0% found this document useful (0 votes)

4 views

Design and Analysis of Algorithm

Uploaded by

jamivamsi4

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

4 views

Design and Analysis of Algorithm

Uploaded by

jamivamsi4

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 77

Introduction to Algorithms

An algorithm is a finite set of instructions that, if followed, accomplishes a particular task.

In addition, all algorithms must satisfy the following criteria :

Input. Zero or more quantities are externally supplied.
Output. At least one quantity is produced.
Definiteness. Each instruction is clear and unambiguous
Finiteness. If we trace out the instructions of an algorithm, then for all cases, the algorithm terminates
after a finite number of steps.
Effectiveness. Every instruction must be very basic so that it can be carried out, in principle, by a
person using only pencil and paper. It is not enough that each operation be definite as in criterion 3; it
also must be feasible.

Process of translating a problem into an algorithm

Sum of n no’s

Abstract solution:
Sum of n elements by adding up elements one at a time.

Little more detailed solution:

Sum=0;
add a[1] to a[n] to sum one element at a time.
return Sum;

Find largest among list of elements

Abstract solution:
Find the Largest number among a list of elements by considering one element at a
time.
Little more detailed solution:
1. Treat first element as largest element
2. Examine a[2] to a[n] and suppose the largest element is at a [ i ];
3. Assign a[i] to largest;

https://www.freshersnow.com/
4. Return largest;

Algorithm Largest (a,n)

{
largest := a[1];
for i := 2 to n do
{
if ( a[i] > largest ) largest := a[i];
}
Return largest;
}

Linear search
Abstract solution:
From the given list of elements compare one by one from first element to last
element for the required element

Little more detailed solution:

for i := 1 to n do
{
Examine a[i] to a[n] and suppose
the required element is at a [ j ];
Write match is found and return position j;
}

Little more detailed solution:

for i := 1 to n do
{
Examine a[i] to a[n] and suppose
the smallest element is at a [ j ];
Interchange a[i] and a[j];
}

Little more detailed solution:

for i := 1 to n do
{
Compare a[1] to a [n-i] pair-wise (each element with its next)
and interchange a[j] and a[j+1] whenever a[j] is greater than a[j+1]
}

Little more detailed solution:

Sort a[1] and a [2]
For i from 3 to n do
{
Suppose a[k] (k between 1 and i) is such that a[k-1] <= a[i] <= a[k].
Then, move a[k] to a[i-1] by one position and insert a[i] at a[k].
}

Recursion has most common applications in Mathematics and computer science.

Example2:
The following definition for natural numbers is Recursive.
1 is a Natural Number.
If k is a natural number then k+1 is a Natural Number.

DEFINITION(Computer Science): This is the method of defining a function in which the function
being defined is applied within its own definition.
In simple terms if a function makes a call to itself then it is known as Recursion.
Example3: Observe the recursive definition for a function Suml(n) to compute
Sum of first n natural numbers.
If n=1 then
return 1
else
return n + Sum(n-1)

https://www.freshersnow.com/
Steps to be followed to write a Recursive function

1.BASIS: This step defines the case where recursion ends.

2.RECURSIVE STEP: Contains recursive definition for all other cases(other than base case)to reduce
them towards the base case

For Example in Example 3,

Basis is n=1 and

Recursive step is n + Sum(n-1)

Note: if any one of the two or both, wrongly defined makes the program infinitely Recursive.

Recursive solution for the Sum of nos

First Call for Recursion:

RecursiveSum (a,n);

Algorithm RecursiveSum(a,k)
{
if (k<=0) then
Return 0.0;
else
Return (RecursiveSum(a,k-1) + a[k]);
}

Recursive solution for the Largest of ‘n’ numbers

First Call for Recursion:

RecursiveLargest (a,n);

Algorithm RecursiveLargest (a,k)

{
if (k = 1) then Return a[k];
else
{
x := RecursiveLargest (a,k-1);
if (x > a[k]) Return x;
else Return a[k];
}
}

Recursive solution for the SelectionSort

First call of recursion is:

https://www.freshersnow.com/
RecursiveSelectionSort (a,1);

Algorithm RecursiveSelectionSort (a, m)

{
if (m = n) Return; // Do nothing
else
{
min:=m;
for k := m + 1 to n do
if (a[k]< a[min]) then min:=k;
//swap
t :=a[m]; a[m] := a[min];a[min]:= t;
RecursiveSelectionSort (a, m+1);
}
}

Recursive solution for the Linear Search

Algorithm linearSearch(a, key, size)

{
if (size == 0) then return -1;
else if (array[size ] = key) return size ;
else return linearSearch (array, key, size - 1);
}

Recursive solution for the BubbleSort

Algorithm RecursiveBubbleSort (a,k)

{
if (k = 1)
Return; // Do nothing
else
{
for j :=1 to k-1 do
{
if ( a[j] > a[j+1] )
{
// swap a[j] and a[j+1]
temp := a[j]; a[j] := a[j+1];
a[j + 1] := temp;
}
}
RecursiveBubbleSort (a,k-1);
}
}

https://www.freshersnow.com/
Recursive solution for the InsertionSort
First call of recursion is:
RecursiveInsertionSort(a,n);

Algorithm RecursiveInsertionSort(a,k)
{
if (k > n)
Return; // Do nothing
else
{
RecursiveInsertionSort(a,k-1);
//Insert 'k'th element into k-1 sorted list
value := a[k]; j := k - 1;
done := false;
repeat
if a[j] > value
{
a[j + 1] := a[j]; j := j - 1;
if (j < 0) done := true;
}
else done := true;
until done;
a[j + 1] := value;
}
}

Recursive solution for the Printing an array in normal order

First call of recursion is:

Recursive Prin_arr(a,n-1);

Recursive Algorithm prin_arr(int a[],int n)

{
if(n=-1)
return;
else
prin_arr(a,n-1);
write a[n]);
}

Recursive solution for the Decimal to binary conversion

Algorithm bin(int n)
{
if (n=1 or n=0)
{
write n;

https://www.freshersnow.com/
return;
}
else
bin=n/2;
write n mod 2;
}

Recursive solution for the Fibonacci number

Recursive Algorithm fib(int x)

{
if(x=0 or x=1) then
return 0;
else
{
if(x=2)
return 1;
else
{
f=fib(x-1) + fib(x-2);
return(f);
}
}

Recursive solution for the Factorial of a given number

Recursive Algorithm fact (int n)

{
{
int fact;
if n=1 tehn
return 1;
else
fact=n*(n-1);
}
return fact;
}

Towers of Hanoi with recursion:

Rules:
(1) Can only move one disk at a time.
(2) A larger disk can never be placed on top of a smaller disk.
(3) May use third post for temporary storage.

Algorithm TowersOfHanoi (numberOfDisks, x,y,z)

{
if (numberOfDisks >= 1)

https://www.freshersnow.com/
{
TowersOfHanoi (numberOfDisks -1, x,z, y);
move (start, end);
TowersOfHanoi (numberOfDisks -1, z, y, x);
}
}

Performance Analysis

Space Complexity:
Algorithm abc (a,b,c)
{
return a+b++*c+(a+b-c)/(a+b) +4.0;
}

 The Space needed by each of these algorithms is seen to be the sum of the following
component.

1. A fixed part that is independent of the characteristics (eg: number, size)of the inputs and
outputs. The part typically includes the instruction space (ie. Space for the code), space for
simple variable and fixed-size component variables (also called aggregate) space for constants,
and so on.
2. A variable part that consists of the space needed by component variables whose size is
dependent on the particular problem instance being solved, the space needed by referenced
variables (to the extent that is depends on instance characteristics), and the recursion stack
space.
 The space requirement s(p) of any algorithm p may therefore be written as,
S(P) = c+ Sp(Instance characteristics)
Where ‘c’ is a constant.
Example 2:
Algorithm sum(a,n)
{
s=0.0;
for I=1 to n do
s= s+a[I];
return s;
}
 The problem instances for this algorithm are characterized by n, the number of elements to
be summed. The space needed d by ‘n’ is one word, since it is of type integer.
 The space needed by ‘a’ a is the space needed by variables of type array of floating point
numbers.
 This is at least ‘n’ words, since ‘a’ must be large enough to hold the ‘n’ elements to be
summed.
 So, we obtain Sum(n)>=(n+s)
[ n for a[ ],one each for n, I a& s]

Alogorithm RSum(a,n)
{

https://www.freshersnow.com/
If(n<= 0) then
return 0.0;
Else
return RSum(a,n-1)+a[n];
}
Recursive function for Sum

Time Complexity:
A Program step is loosely defined as a syntactically or semantically meaningful segment of a
program that has an execution time that is independent of the instance characteristics.
For Ex: Return a+b+b*c+(a+b-c)/(a+b)+4.0;

Program step count method – examples:

Algorithm Sum (a,n)
{
s:= 0.0;
Count :=Count+=1; // count is global;it is initially zero.
for i:=1 to n do
{
Count :=count+1 //for for
S :=S+a[i];count:=count+1;For assignment
}
Count:=count+1; // For last time of for
Count:=count +1; //For the return
return s;
}
Algorithm SUM with count statements added

Algorithm Sum (a,n)

{
for i:=to n do count:=count+2;
count :=count +3;
}
Algorithm for Sum Simplified version for previous sum

Algorithm RSum(a,n)
{
Count :=count+1; // For the if conditional
if (n < 0)then
{
count:=count+1//For the return
return 0.0;
}
else

https://www.freshersnow.com/
{
count :=count+1 // For the addition, function invocation and return
return RSum(a,n-1)+a[n];
}
}
Algorithm Recursive sum with count statements added

Algorithm Add (a,b,c,m,n)

{
for i:= 1to m do
for j:=1 to n do
c[i,j] := a[i,j] + b[i,j];
}
Matrix Addtion

Algorithm Add(a,b,c,m,n)
{
for i:=1 to m do
{
count :=count +1; // For ‘for i’
for j :=1 to n do
{
count :=count+1; // For ‘for j’
c[i, j] := a[i, j]+b[i, j];
count :=count +1; //For the assignment
}
count := count +1 ;// For loop initialization and last time of ‘for j’
}
count := count+1 ; // For loop initialization and last time of ‘for i’
}
Algorithm for Matrix addition with counting statements

Step table method:

Statement s/e frequency total steps

1 .Algorithm Sum(a,n) 0 ------ 0
2 { 0 ------ 0
3 s:=0.0; 1 1 1
4 for i := 1to n do 1 n+1 n+1
5 s := s + a[i]; 1 n n
6 return 1 1 1
7 } 0 ----- 0
TOTAL 2n +3
Step table for Algorithm finding Sum

https://www.freshersnow.com/
Statement s/e Frequency total steps
n=0 n>0 n=0 n>0
1.Algorithm RSum(a,n) 0 - - 1. 0
2.{
3. if(n< 0) then 1 1 1 1 1
4. return 0.0: 1 1 0 1 0
5. else return
6.RSum (a,n-1) + a[n]; 1+x 0 1 0 1+x
7. } 0 --- --- 0 0
Total 2 2+x

x = tRSum (n-1)

Step table for Algorithm RSum

Statement s/e frequency Total steps

1. Algorithm Add(a,b,c,m,n) 0 ----- 0
2. { 0 ----- 0
3. for i := 1 to m do 1 m+1 m+1
4. for j := 1 to n do 1 m(n+1) mn + m
5. c[i,j] :=a[i,j] + b[i,j]; 0 mn mn
6. } ------ 0
Total 2mn +2m +1

Step table for Algorithm Addition of two matrices

Statement s/e frequency Total steps

Algorithm BubbleSort (a,n)
{
for i := 1 to n do
{
for j :=1 to n-i do
{
if ( a[j] > a[j+1] )
{
// swap a[j] and a[j+1]
temp := a[j];
a[j] := a[j+1];
a[j + 1] := temp;
}
}
}
}
Total
Fill the blank columns in the above Table

https://www.freshersnow.com/
Statement s/e frequency Total steps
Algorithm Linear search (a, req_ele)
{
Flag=0
for i := 1 to n do
{
If (a[i] = req_ele) then
{
Flag=1;
Return i;
}
}
If (flag) then
Write “element found at i position”
Else
Write “element not found
}
Total

Fill the blank columns in the above Table

https://www.freshersnow.com/
Kinds of Analysis of Algorithms
• The Priori Analysis is aimed at analyzing the algorithm before it is implemented(based on the
algorithm) on any computer. It will give the approximate amount of resources required to solve the
problem before execution. In case of priori analysis, we ignore the machine and platform dependent
factors. It is always better if we analyze the algorithm at the earlier stage of the software life cycle.
Priori analysis require the knowledge of

– Mathematical equations
– Determination of the problem size
– Order of magnitude of any algorithm

• Posteriori Analysis is aimed at determination of actual statistics about algorithm’s consumption of

time and space requirements (primary memory) in the computer when it is being executed as a program
in a machine.
Limitations of Posteriori analysis are
• External factors influencing the execution of the algorithm
– Network delay
– Hardware failure etc.,
• The information on target machine is not known during design phase
• The same algorithms might behave differently on different systems
– Hence can’t come to definite conclusions

Asymptotic notations(O,Ω,Ө)
Step count is to compare time complexity of two programs that compute same function and also to
predict the growth in run time as instance characteristics changes. Determining exact step count is
difficult and not necessary also. Since the values are not exact quantities we need only comparative
statements like c1 n2 ≤ tp(n) ≤ c2n2.

For ex: consider two programs with complexities c1n2 + c2 n and c3 n respectively. For small values
of n, complexity depend upon values of c1, c2 and c3. But there will also be an n beyond which
complexity of c3 n is better than that of c1 n2 + c2 n.This value of n is called break-even point. If this
point is zero, c3n is always faster (or at least as fast).

c1=1,c2=2 & c3=100

Then c1n2+c2 n is ≤ c3n for n≤98 and

c1n2+c2n is > c3n for n>98

The Common asymptotic functions are given below.

https://www.freshersnow.com/
Function Name
1 Constant
log n Logarithmic
N Linear
n log n n log n
n2 Quadratic
n3 Cubic
2n Exponential
n! Factorial

The growth of the functions as below

1< log n < n <n log n < n2 < n3 < 2n <n!

Definition [Big ‘oh’] The function f(n)=O(g(n)) iff there exist positive constants c and no such that
f(n)≤c*g(n) for all n, n ≥ no.

Ex1: f(n) = 2n + 8, and g(n) = n2. Can we find a constant c, so that 2n + 8 <= n2? The number 4 works
here, giving us 16 <= 16.
For any number c greater than 4, this will still work. Since we're trying to generalize this for large
values of n, and small values (1, 2, 3) aren't that important, we can say that f(n) is generally faster than
g(n); that is, f(n) is bound by g(n), and will always be less than it.

Ex2: The function 3n+2=O(n) as 3n+2≤4n for all n≥2.

Pb1: 3n+3=O(_) as 3n+3≤ for all____.

Ex3: 10n2+4n+2=O(n2) as 10n2+4n+2≤11n2 for all n≥5

Pb2:1000n2+100n-6=O(_) as 1000n2+100n-6≤for all_

Ex4:62n+n2=O(2n) as 62n+n2 ≤ 7*2n for n ≥ 4

Ex5:3n+3=O(n2) as 3n+3≤3n2 for n≥2.

Ex 6:10n2+4n+2=On4 as10n2+4n+2≤10n4 for n≥2.

Ex7:3n+2≠O(1) as3n+2 not less than or equal to c for any constant c and all n≥n0

Ex 8: 10n2+4n+2≠O(n)

Definition[Omega] The function f(n) =Ώ (g(n) (read as “f of n is omega of g of n'') iff there exist
positive constants c and n0 such that f(n) ≥c *g(n) for all n, n≥ no.

https://www.freshersnow.com/
Ex1: The function 3n+2=Ω(n) as 3n+2≥3n for n≥1
(the inequality holds for n≥0,but the definition of Ω requires an n0>0).
3n+2=Ω(n) as 3n+2≥3n for n≥1.

Ex:2 100n+6=Ω(n) as 100n+6≥100n for n≥1

Pb:1 6n+4= Ω(_____) as 6n+4 ≥_______ for all n≥ _____
Ex:3 10n2 +4n+2= Ω(n2) as 10n2 +4n+2 ≥ n2 for n ≥ 1
Pb:2 2n2+3n+1= Ω( _____ ) as 2n2+3n+1 ≥_______ for all n≥_________
Ex: 4 6* 2n + n2 = Ω(2n) as 6* 2n + n2 ≥ 2n for n ≥ 1.

Pb:3 4n2 – 64 n + 288 = Ω ()as 4n2 – 64 n + 288≥_ for all n≥_______ .

Pb:5 n3 logn is = Ω ()as n3 logn____for all n≥ _.

Definition [Theta] The function f(n) = Θ (g(n) ) (read as “f of n is theta of g of n'') iff there exist
positive constants C1,C2, and n0 such that c1g(n)≤ f(n) ≤ c2g(n) for all n, n≥n0

Ex 1 : The function 3n + 2 = Ө(n) as 3n + 2 ≥ 3n for all n ≥ 2 and 3n + 2 ≤ 4n for all n ≥ 2 so c1 = 3,

c2=4 and n0 =2

Pb 1: 6n+4= Θ(___) as 6n+4≥________for all n ≥_______and 6n+4 ≤ _______ for all n ≥_______
Ex 2 : 3n + 3 = Ө(n)
Ex 3 : 10n2 + 4n +2 = Ө(n2)
Ex 4: 6* 2n + n2 = Θ (2n)
Ex5:10*log n+4= Θ (log n)
Ex6:3n+2≠ Θ(1)
Ex7:3n + 3 = Ө(n)
Ex 8: 10n2 +4n+2 ≠ Θ (n)

Theorem: If f(n) = amnm+.......................+a3n3+a2n2+a1n+a0, then f(n)=O (nm)

i
Proof: f(n) ≤ ∑mi = 0 | ai | n
m i-m
≤n ∑mi = 0│ai│n
m
≤ n ∑mi = 0│ai│ for n≥1

f(n) = O(nm) (assuming that m is fixed ).

https://www.freshersnow.com/
Theorem: If f(n) = am nm ++................+a3n3+a2n2+a1 n + ao and am > 0, then f(n)= Ω(nm)
proof : Left as an exercise.

Definition [Little ‘oh’] The Function f (n) =o(g(n)) (read as ‘f of n is little oh of g of n’)iff

lim f(n) =0
n→∞ g(n)

Example:

The function 3n+2 = o(n2) since lim 3n+2 = 0.

n-->∞ n2

Ex1: 3n+2 =o(n log n).

Ex:2 3n+2 =o(n log log n).
Ex:3 6*2n +n2 =o(3n).
Ex:4 6 * 2n+n2 =o(2nlog n).
Ex:5 6 * 2n+n2 ≠o(2n).

Analogous to ‘o’ notation ‘ ω ‘notation is defined as follows.

Definition [Little omega] The function f(n)= ω(g(n)) (read as “ f of n is little omega of g of n”) iff
lim g(n) = 0
n→∞ f(n)

https://www.freshersnow.com/
Divide and Conquer
Divide and Conquer for finding Counterfeit coin (exactly one coin):
Algorithm CounterfeitCoin(Bunch, numberofcoins)
{
if (numberofcoins = 2) {
weigh the two coins;
if one of them weighs less that is counterfeit coin;
else, no counterfeit coin in the Bunch
}
else {
Divide Bunch into two halves Bunch1 and Bunch2;
if Bunch1 weighs less than Bunch 2, call CounterfeitCoin(Bunch1, numberofcoins / 2);
else, call CounterfeitCoin(Bunch2, numberofcoins / 2);
}
}

Finding Largest with Divide and Conquer:

Algorithm DandCLargest (a,k, j)
{
if (k = j) Return a[n];
else
{
mid := (k + j) /2;
x1= DandCLargest (a,k,mid);
x2= DandCLargest (a,mid+1,j);
if (x1 > x2) Return x1;
else Return x2;
}
}

First Call for Recursion:

DandCLargest (a,1,n);

Divide and Conquer algorithm design works on the principle of dividing the given problem into
smaller sub problems which are similar to the original problem. The sub problems are ideally of
the same size.

The Divide and Conquer strategy can be viewed as one which has three steps. The first step is
called Divide which is nothing but dividing the given problems into smaller sub problems which
are identical to the original problem and also these sub problems are of the same size. The
second step is called Conquer where in we solve these sub problems recursively. The third step

https://www.freshersnow.com/
is called Combine where in we combine the solutions of the sub problems to get the solution for
the original problem.

Control Abstraction for Divide and Conquer Algorithms – General Method

Algorithm DandC(P) {
if Small(P) then return S(P);
else {
Divide P into smaller instances P1,P2,…….,Pk, k≥1;
Apply DandC to each of these sub-problems;
Return Combine(DandC(P1),DandC(P2)...,DandC(Pk)),
}
}
Binary Search
• Binary Search takes a sorted array as the input
• It works by comparing the target (search key) with the middle element of the array and
terminates if it equals, else it divides the array into two sub arrays and continues the search in left
(right) sub array if the target is less (greater) than the middle element of the array.
Reducing the problem size by half is part of the Divide step in Binary Search and searching in
this reduced problem space is part of the Conquer step in Binary Search. There is no Combine
step in Binary Search.

With Divide and Conquer (recursion)

Algorithm BinSrch(a,i,l,x) {
// Given an array a[I:l] of elements in non decreasing order, 1≤ i ≤ l,
// determine whether x is //present and if so, return j such that x =a[j]; else return 0.
if (l=i) then { // if Small(P)
if (x=a(i)) then return i; else return 0;
}
else {
// Reduce P into a smaller subproblem.
mid :=[(i + l)/2 ];
if (x=a[mid]) then return mid;
else if (x<a[mid] then
return BinSrch (a,i, mid -1,x);
else return BinSrch (a, mid +1,l,x);
}
}
Iterative Binary Search (non-recursive)

Algorithm BinSearch (a,n,x) {

low :=1;high :=n;
while (low ≤ high) do {
mid :=(low +high)/2;

https://www.freshersnow.com/
if ( x<a[mid]) then high :=mid – 1;
else if (x>a[mid]) then low :=mid + 1;
else return mid;
}
return 0;
}

Theorem Algorithm BinSearch(a,n,x) works correctly.

Proof:
We assume that all statements work as expected and that comparisons such as x>a[mid] are
appropriately carried out.
Initially low = 1, high = n, n ≥ 0, and a[1]≤ a[2]≤..≤ a[n].
If n = 0, the while loop is not entered and 0 is returned.
Otherwise we observe that each time through the loop the possible elements to be checked for equality
with x are a [low], a[low + 1]…, ..., a[mid],..., a[high).
If x = a[mid], then the algorithm terminates successfully.
Otherwise the range is narrowed by either increasing low to mid + 1 or decreasing high to mid — 1.
Clearly this narrowing of the range does not affect the outcome of the search.
If low becomes greater than high, then x is not present and hence the loop is exited.

Performance of Binary Search

Theorem:-If n is in the range [2k_1, 2k), then BinSearch makes at most k element comparisons
for a successful search and either k —1 or k comparisons for an unsuccessful search. (In other
words the time for a successful search is 0 (log n) and for an unsuccessful search is (log n).

Proof: Consider the binary decision tree describing the action of BinSearch on n elements. All
successful searches end at a circular node whereas all unsuccessful searches end at a square node.
If 2 k-1 ≤ n <2k, then all circular nodes are at levels 1, 2,... , k whereas all square nodes are at levels k
and k + 1 (note that the root is at level 1). The number of comparisons needed to terminate at a
circular node on level i is i whereas the number of element comparisons needed to terminate at a
square node at level i is only i — 1. The theorem follows.

Merge Sort

Merge sort is yet another sorting algorithm which works on the Divide and Conquer design
principle.
• Merge sort works by dividing the given array into two sub arrays of equal size
• The sub arrays are sorted independently using recursion
• The sorted sub arrays are then merged to get the solution for the original array.

https://www.freshersnow.com/
The breaking of the given input array into two sub arrays of equal size is part of the Divide step.
The recursive calls to sort the sub arrays are part of the Conquer step. The merging of the sub
arrays to get the solution for the original array is part of the Combine step.

The basic operation in Merge sort is comparison and swapping. Merge Sort Algorithm calls it
self recursively. Merge Sort divides the array into sub arrays based on the position of the
elements whereas Quick Sort divides the array into sub arrays based on the value of the
elements. Merge Sort requires an auxiliary array to do the merging (Combine step). The
merging of two sub arrays, which are already sorted, into an auxiliary array can be done in O(n)
where n is the total number of elements in both the sub arrays. This is possible because both the
sub arrays are sorted.

Algorithm MergeSort(low, high) {

// Small(P) is true if there is only one element to sort .
// In this case the list is already sorted.
if (low < high) then {
//If there are more than one element
//Divide P into subproblems.
mid := [(low + high)/2] ;
// Solve the subproblems.
MergeSort(low, mid);
MergeSort (mid + 1, high);
// Combine the solutions.
Merge(low, mid, high);
}
}

Merging in Merge Sort

Algorithm Merge (low, mid, high) {

// a[low:high] is a global array containing two sorted subsets in
// a [low:mid] and in a [mid + 1 :high]. The goal is to merge these
// two sets into a single set residing in a [low:high]. B[] is an
// auxiliary global array.
h:=low; i:=low ; j : =mid +1;
while ((h≤ mid) and ( j≤high)) do {
if (a[h]≤a[j] then {
b[i] := a[h];h :=h+1;
}
else {
b[i] := a[j];j :=j+1;
}
i:= i+1;
}
if ( h>mid) then

https://www.freshersnow.com/
for k:=j to high do
{ b[i] : =a[k]; i:=i+1; }
else for k :=h to mid do
{ b[i] := a[k]; i:= i+1; }
for k: =low to high do a[k] :=b[k];
}

Complexity of Merge Sort is O(n log n) and binary search is O(log n).

This can be proved by repeated substitution in the recurrence relations.

Suppose (for simplicity) that n = 2k for some entire k. as n=2k k = log2n

Merge Sort:
Let T(n) the time used to sort n elements. As we can perform separation and merging in linear time, it
takes cn time to perform these two steps, for some constant c. So, recurrence relation is : T(n) =
2T(n/2) + cn.
In the same way: T(n/2) = 2T(n/4) + cn/2, so T(n) = 4T(n/4) + 2cn.
Going in this way ...
T(n) = 2mT(n/2m) + mcn, and

T(n) = 2kT(n/2k) + kcn = nT(1) + cnlog2n = O(n log n).

Binary Search:
Let T(n) the time used to search n elements. As we need to search only one of the halves, the
Recurrence relation is : T(n) = T(n/2) + c
In the same way: T(n/2) = T(n/4) + c, so T(n) = T(n/4) + 2c.
Going in this way ...
T(n) = T(n/2m) + mc, and

T(n) = T(n/2k) + kc = T(1) + kc = kc + 1 = O(log n).

QuickSort:

Quick sort is one of the most powerful sorting algorithms. It works on the Divide and Conquer
design principle. Quick sort works by finding an element, called the pivot, in the given input
array and partitions the array into three sub arrays such that the left sub array contains all
elements which are less than or equal to the pivot. The middle sub array contains the pivot. The

https://www.freshersnow.com/
right sub array contains all elements which are greater than the pivot. Now, the two sub arrays,
namely the left sub array and the right sub array are sorted recursively.
The partitioning of the given input array is part of the Divide step. The recursive calls to sort
the sub arrays are part of the Conquer step. Since the sorted sub arrays are already in the right
place there is no Combine step for the Quick sort.

1. Algorithm QuickSort (p,q)

2. // Sorts the elements a[p],…, a[q] which reside in the global
3. // array a[1 ;n] into ascending order ; a[n+1] is considered to
4. // be defined and must be ≥ all the elements in a[1 :n]
5. {
6. if (p < q) then // if there are more than one element
7. {
8. // divide P into two sub problems.
9. j : = Partition (a,p,q +1);
10. // j is the position of the partitioning element.
11. // Solve the subproblems.
12. QuickSort (p,j -1),
13. QuickSort ( j +1,q)
14. // there is no need for combining solutions.
15. }
16. }

Algorithm for Sorting by partitioning.

1. Algorithm Partition (a,m,p)

2. //Within a[m],a[m+1],….a[p-1] the elements are
3. //rearranged in such a manner that if initially t = a[m],
4. // then after completion a[q] = t for some q between m
5. // and p-1, a[k]≤ t for m≤ k<q, and a [k] ≥ t
6. // for q< k<p. q is returned. Set a[p] = ∞.
7. {
8. v :=a[m], I :=m; j : = p;
9. repeat
10. {
11. repeat
12. i :=i+1;
13. until (a[i]≥v);

14. repeat
15. j : = j – 1;
16. until (a[j] ≤ v);

17. if (I < i) then interchange (a,I,j);

https://www.freshersnow.com/
18. } until (I ≥ j);

19. a[m] : = a[j]; a[j] : = v; return j ;

20. }

1. Algorithm Interchange ( a , i, j)
2. //Exchange a[i] with a [j]
3. {
4. p : = a[i];
5. a [i] : = a[j]; a[j] : p;
6. }

Algorithm Partition the array a[m : p-1] about a [m]

(1) (2) (3) (4) (5) (6) (7) (8) (9) (10) i (p)

65 70 75 80 85 60 55 50 45 +∞ 2 9

65 45 75 80 85 60 55 50 70 +∞ 3 8

65 45 50 80 85 60 55 75 70 +∞ 4 7

65 45 50 55 85 60 80 75 70 +∞ 5 6

65 45 50 55 60 85 80 75 70 +∞ 6 5

60 45 50 55 65 85 80 75 70 +∞

https://www.freshersnow.com/
Greedy Algorithms
Greedy algorithms – overview

Greedy design technique is primarily used in Optimization problems.

Optimization problems are problems where in we would like to find the best of all possible
solutions. In other words, we need to find the solution which has the optimal (maximum or
minimum) value satisfying the given constraints.

The Greedy approach helps in constructing a solution for a problem through a sequence of steps
where each step is considered to be a partial solution. This partial solution is extended
progressively to get the complete solution.

In the greedy approach each step chosen has to satisfy the constraints given in the problem. Each
step is chosen such that it is the best alternative among all feasible choices that are available. The
choice of a step once made cannot be changed in subsequent steps.

Change making example:

Suppose, we want to make change for an amount ‘A’ using fewest no of currency notes. Assume
the available denominations are Rs 1, 2, 5, 10, 20, 50, 100, 500, 1000.
To make a change for A=Rs 28, with the minimum number of notes, one would first choose a
note of denomination Rs 20, 5, 2 and 1.

Denomination table
for Rs 28 for Rs 783 for Rs 3799
1000 X 0 0 1000 X 1000 X
500 X 0 0 500 X 500 X
100 X0 0 100 X 100 X
50 X 0 0 50 X 50 X
20 X 1 20 20 X 20 X
10 X 0 0 10 X 10 X
5X1 5 5X 5X
2X1 2 2X 2X
1X1 1 1X 1X
Total 28 Total Total

Algorithm change making(denom_value[ ], TargetAmount)

{
// denom={1000, 500, 100, 50, 20, 10, 5, 2, 1}

selected amount (SA) :=0;

i :=1;
while (SA < TargetAmount){
if (SA + denom_value [i] <= TargetAmount ) { // Select & Feasible

denom_select[i] ++; // Union

https://www.freshersnow.com/
SA := SA + denom_value[i];
}
else {
i++;
}
}
Print denom _select
}

Greedy Algorithm-General method

In Greedy method the problems have 'n' inputs called as candidate set, from which a subset is
selected to form a solution for the given problem. Any subset that satisfies the given constraints
is called a feasible solution. We need to find a feasible solution that maximizes or minimizes an
objective function and such solution is called an optimal solution.

In the above ex currency notes denomination set { 1000 ….1000 ,500….500, 100….100,
50…50, 20…20,10…10,5…5,2..2,1…1}is candidate set.

In the above ex constraint is our solution make the exact target amount of cash. Hence, any
feasible solution i.e. sum of selected notes should be equal to target amount.

In the above ex objective function is our solution should consist of the fewest number of
currency notes. Hence, any optimal solution which is one of the feasible solutions that optimizes
the objective function. There can be more than one optimal solution.

Control Abstraction for Greedy General Method

Algorithm Greedy (a, n)

// a [1 ..n] contains the n inputs
{
solution :=ø;
for i := 1 to n do
{
x := Select (a);
if Feasible (solution, x) then solution := Union ( solution, x);
}
return solution;
}
Greedy method consists of 3 functions (steps).

1) Select: it selects an input from array a[ ] (candidate set) and puts in the variable x.

2) Feasible: it is a Boolean function which checks whether the selected input meets the
constraints or not.

3) Union: if the selected input i.e. 'x' makes the solution feasible, then x is included in the
solution and objective function get updated.

https://www.freshersnow.com/
Characteristics of Greedy:

1) These algorithms are simple and straightforward and easy to implement.

2) They take decisions on the basis of information at hand without worrying about the effect
these decisions may have in the future.

3) They work in stages and never reconsider any decision.

Greedy Algorithms Applications

Knapsack problem:

A thief robbing a store finds n items, the items each worth vi rupees and weights wi grams,
where vi and wi are positive numbers. He wants to take as valuable load as possible but he
can carry at most w grams in his knapsack(bag). Which item should he take?

They are two types of knapsack problem.

1) 0-1 knapsack problem: Here the items may not be broken into smaller pieces, so thief may
decide either to take an item or to leave to it(binary choice). It cannot be efficiently solved by
greedy algorithm

2) Fractional (General) Knapsack problem: Here thief can take the fraction of items, meaning
that the items can be broken into smaller pieces so that thief may be able to carry a fraction x i of
item i. This can be solved easily by greedy.

If a fraction xi , 0 ≤ xi ≤ 1, of objects i is placed into the knapsack, then a profit of pi xi is earned.

The objective is to obtain a filling of the knapsack that maximizes the total profit earned. Since
the knapsack capacity is m, we require the total weight of all chosen objects to be at most m.

Formally the problem can be stated as

Maximize ∑ Pi xi …………………………………..(1)

Subject to ∑ Wi xi ≤ m………………………………………..(2)

And 0 ≤ xi ≤1, 1≤ i ≤n………………………………………..(3)

The profit and weights are positive numbers. A feasible solution is any set
(x1,x2,………………..xn) satisfying (2) and(3). An optimal solution is feasible solution for which
(1) is maximized.

Eg; consider the following instance of the knapsack problem.

n=3, m=20 , (P1, P2, P3) =(25, 24, 15) & (w1, w2,w3) = (18,15,10)

Note that knapsack problem calls for select a subset of the objects hence fits the subset paradigm.

https://www.freshersnow.com/
1) We can try to fill the knapsack by including the object with largest profit(greedy approach to
the profit) .If an object under consideration does not fit, then a fraction of it is included to fit the
knapsack. Object 1 has the largest profit value.P1=25. So it is placed into the knapsack first. Then
x1=1 and a profit of 25 is earned. Only 2 units of knapsack capacity are left. Objects 2 has the
next largest profit P2=24. But W2 =15 & it does not fit into the knapsack. Using x2 =2/15 fills the
knapsack exactly with the part of the object 2.

The method used to obtain this solution is termed a greedy method at each step, we chose to
introduce that object which would increase the objective function value the most.
(x1 , x2 , x3) ∑ wi xi ∑ pi xi
(1 , 2/15 , 0) 20 28.2
This is not an optimal solution.
2)We apply greedy approach by choosing value per unit weight is as high as possible
Item(n) Value(p1,p2,p3) Weight(w1,w2,w3) Val/weight

1 25 18 1.388

2 24 15 1.6

3 15 10 1.5

Here p2/w2 > p3/w3 > p1/w1. Now the items are arranged into non increasing order of p i/wi.
Second item is the most valuable item. We chose item 2 first. Item 3 is second most valuable
item. But we cannot choose the entire item3 as the weight of item 3 exceeds the capacity of
knapsack. We can take ½ of the third item. Therefore the solution is x1 = 0, x2 = 1, x3 = ½ and
maximum profit is ∑ pixi = 0*25 + 1*24 + ½ * 15 = 31.5

(x1, x2, x3) ∑wixi ∑pixi

(1, 2/15, 0) 20 28.2
(0, 2/3, 1) 20 31
(0, 1, 1/2) 20 31.5

If the items are already arranged in non increasing order of pi/wi , then the function
greedy knapsack obtains solution corresponding to this strategy.

Algorithm Greedy Knapsack(a,n)

// Objects are sorted in the non-increasing order of p[i]/w[i]
{
for i := 1 to n do x[i] := 0.0;
U := m;
for i := 1 to n do
{
if (w[i] > U ) then break;
x[i] := 1; U:=U - w[i];
}

https://www.freshersnow.com/
if (i <= n) then x[i] := U / w[i];
}

Analysis of Greedy Knapsack

 If the items are already sorted into decreasing order of vi/wi, then time complexity is
O(n)
 Therefore Time complexity including sort is O(n log n)

Minimum Spanning Tree

A tree is defined to be an undirected, acyclic and connected graph (or more simply, a
graph in which there is only one path connecting each pair of vertices).

Assume there is an undirected, connected graph G. A spanning tree is a sub-graph of G,

is a tree, and contains all the vertices of G. A minimum spanning tree is a spanning tree,
but has weights or lengths associated with the edges, and the total weight of the tree (the
sum of the weights of its edges) is at a minimum

Application of MST

1) practical application of a MST would be in the design of a network. For instance, a group
of individuals, who are separated by varying distances, wish to be connected together in a
telephone network. MST can be used to determine the least costly paths with no cycles in
this network, thereby connecting everyone at a minimum cost.

2) Another useful application of MST would be finding airline routes MST can be applied to
optimize airline routes by finding the least costly paths with no cycles

https://www.freshersnow.com/
28
1 1
2 10 2
10 14 16
16
14
6 3 6 7 3

24 7
25 18 25 5 12
12
22 4
5
22 4

(a) (b)

Prim’s Algorithm (DJP algorithm,the Jarník algorithm, or the Prim-Jarník algorithm).

Prim's algorithm finds a minimum spanning tree for a connected weighted graph. This means
it finds a subset of the edges that forms a tree that includes every vertex, where the total
weight of all the edges in the tree is minimized.

Steps

 Builds the tree edge by edge

 Next edge to be selected is one that result in a minimum increase in the sum of costs
of the edges so far included
 Always verify that resultant is a tree

Ex:

Consider the connected graph given below

https://www.freshersnow.com/
Minimum spanning tree using Prim’s algorithm can be formed as follows.

Algorithm Prim(E,cost,n,t)

//E is the set of edges in G.cost [1 :n,1:n] is the cost

//adjacency matrix of an n vertex graph such that cost[i, j]is
//either a positive real number or ∞ if no edges( i, j) exists,
//A minimum spanning tree is computed and stored as set of
//edges in the array t[1 :n -1,1:2], (t[i,1],t[i,2]) is an edge in
//the minimum –cost spanning tree. The final cost is returned.
{
Let (k,l) be an edge of minimum cost in E;
mincost :=cost[k,l];
t[1,1] :=k;t[1,2] :=l;
for i :=1 to n do //Initialize near.
if (cost[i ,l ]< cost[i ,k ]) then
near[i] :=l;
else
near[i]:=k;
near[k] :=near[l] :=0;
for i := 2 to n-1 do
{ // Find n - 2 additional edges for t.
Let j be an index such that near [j] ≠ 0 and
cost [j,near[j]] is minimum;
t[i,1]:= j; t[i , 2] := near[j];

https://www.freshersnow.com/
mincost := mincost + cost[j,near [j]];
near[j]:=0;
for k:=1 to n do // Update near[ ]
if((near[k]≠0) and (cost[k,near[k]] > cost[k,j]))then
near[k]:=j;
}
return mincost;
}

Time complexity of the above algorithm is O(n2).

Kruskal’s Algorithm
Kruskal's algorithm is another algorithm that finds a minimum spanning tree for a connected
weighted graph. If the graph is not connected, then it finds a minimum spanning forest (a
minimum spanning tree for each connected component).

Kruskal's Algorithm builds the MST in forest. Initially, each vertex is in its own tree in
forest. Then, algorithm considers each edge in turn, order by increasing weight. If an edge (u,
v) connects two different trees, then (u, v) is added to the set of edges of the MST, and two
trees connected by an edge (u, v) are merged into a single tree on the other hand, if an edge
(u, v) connects two vertices in the same tree, then edge (u, v) is discarded. The resultant may
not be a tree in all stages. But can be completed into a tree at the end.

t = EMPTY;
while ((t has fewer than n-1 edges) && (E != EMPTY))
{
choose an edge(v, w) from E of lowest cost;
delete (v, w) from E;
if (v, w) does not create a cycle in t
add (v, w) to t;
else
discard (v, w);
}

To check whether there exist a cycle, place all vertices in the same connected component of t
into a set. Then two vertices v and w are connected in t then they are in the same set.

https://www.freshersnow.com/
Example:

Consider the connected graph given below:

Minimum spanning tree using Kruskal’s algorithm can be formed as given below.

Kruskal’s Algorithm
Float kruskal (int E[ ][ ], float cost[ ][ ], int n, int t[ ][2])
{
int parent[w];
consider heap out of edge cost;
for (i=1; i<=n; i++)
parent[i] = -1; //Each vertex in different set
i=0;
mincost = 0;
while((i<n-1) && (heap not empty))

https://www.freshersnow.com/
{
Delete a minimum cost edge (u,v) from the heap and re heapify;
j = Find(u); k = Find(v); // Find the set
if (j != k)
{
i++;
t[i][1] = u;
t[i][2] = v;
mincost += cost[u][v];
Union(j, k);
}
if (i != n-1)
printf(“No spanning tree \n”);
else
return(mincost);
}
}

Time complexity of the above algorithm is O(n log n)

Greedy Algorithm for Single-source shortest paths to all other vertices

1 2 3
15
35 30
10 20 20

4 5 6
15 3

Path Length

1) 1.4 10
2) 1.4.5 25
3) 1.4.5.2 45
4) 1.3 45

(b) Shortest path from 1

https://www.freshersnow.com/
Greedy algorithm to generate shortest paths

Algorithm Shortest paths (v, cost ,dist, n)

// dist[j]. 1≤ j≤ n, is set to the length of the shortest
// path from vertex v to vertex j in a digraph G with n
// vertices, dist[v]is set to zero. G is represented by its
// cost adjacency matrix cost[1:n ,1;n]
{
for i:=1 to n do
{ // Initialize S.
S[i] :=false; dist[i]:=cost[v,i];
}
S[v] :=true; dist[v] :=0.0; //put v in S.
for num := 2 to n do
{
// Determine n - 1 paths from v.
Choose u from among those vertices not
in S such that dist[u] is minimum;
S[n] :=true;//put u in S.
for (each w adjacent to u with S[w] = false ) do
// Update distances.
if(dist[w] > dist[u] + cost[u,w])) then
dist[w] :=dist[u] + cost[u, w];
}
}

Boston
5
San Francisco Chicago 1500 250
4
2 800 1000 New York
3 6
300 Denver 1400 900
1000
1700
1
8
Los Angeles 1000 7

New orleans Miami

(a) Digraph

https://www.freshersnow.com/
1 2 3 4 5 6 7 8

1. 0
2. 300 0
3. 100 800 0
4. 1200 0
5. 1500 0 250
6. 1000 0 900 1400
7. 0 1000
8. 1700 0
(a) Length – adjacency matrix

https://www.freshersnow.com/
Dynamic Programming (DP)
Dynamic Programming is a design principle which is used to solve problems with
overlapping sub problems.

 It is used when the solution to a problem can be viewed as the result of a sequence of
decisions. It avoids duplicate calculation in many cases by keeping a table of known
results and fills up as sub instances are solved.
 In Dynamic Programming We usually start with the smallest and hence the simplest sub-
instances. By combining their solutions, we obtain the answers to sub-instances of
increasing size, until finally we arrive at the solution of the original instance.
 It follows a bottom-up technique by which it start with smaller and hence simplest sub
instances. Combine their solutions to get answer to sub instances of bigger size until we
arrive at solution for original instance.

How DP differ from Greedy and Divide & Conquer

 Dynamic programming differs from Greedy method because Greedy method makes only
one decision sequence but dynamic programming considers more than one decision
sequence. However, sequences containing sub-optimal sub-sequences can not be optimal
and so will not be generated.

 Divide and conquer is a top-down method. When a problem is solved by divide and
conquer, we immediately attack the complete instance, which we then divide into smaller
and smaller sub-instances as the algorithm progresses. The difference between Dynamic
Programming and Divide and Conquer is that the sub problems in Divide and Conquer
are considered to be disjoint and distinct where as in Dynamic Programming they are
overlapping.

Principle of Optimality

 An optimal sequence of decisions has the property that whatever the initial state and
decisions are, the remaining decisions must constitute an optimal decision sequence with
regard to the state resulting from the first decision.
 In other words, principle of optimality is satisfied when an optimal solution is found for a
problem then optimal solutions are also found for its sub-problems also.

General Method
The Idea of Developing a DP Algorithm

Step1: Structure: Characterize the structure of an optimal solution.

– Decompose the problem into smaller problems, and find a relation between the structure of
the optimal solution of the original problem and the solutions of the smaller problems.
Step2: Principle of Optimality: Recursively define the value of an optimal solution.
– Express the solution of the original problem in terms of optimal solutions for smaller
problems.

https://www.freshersnow.com/
Step 3: Bottom-up computation: Compute the value of an optimal solution in a bottom-up
fashion by using a table structure.
Step 4: Construction of optimal solution: Construct an optimal solution from computed
information.
Steps 3 and 4 may often be combined.
Remarks on the Dynamic Programming Approach
Steps 1-3 form the basis of a dynamic-programming solution to a problem. Step 4 can be
omitted if only the value of an optimal solution is required.

All pairs Shortest Paths

Let G=(V,E) be a directed graph with n vertices.The cost of the vertices are represented by an
adjacency matrix such that 1≤i≤n and 1≤j≤n

cost(i, i) = 0,

cost(i, j) is the length of the edge <i, j> if <i, j>  E and

cost(i, j) = ∞ if <i, j> E.

The graph allows edges with negative cost value, but negative valued cycles are not allowed.
All pairs shortest path problem is to find a matrix A such that A[i][j] is the length of the
shortest path from i to j.

Consider a shortest path from i to j, i≠j. The path originates at i goes through possibly many
vertices and terminates at j. Assume that there is no cycles. If there is a cycle we can remove
them without increase in the cost because there is no cycle with negative cost.

Initially we set A[i][j] = cost (i,j).

Algorithm makes n passes over A. Let A0, A1, .. An represent the matrix on each pass.

Let Ak-1[i,j] represent the smallest path from i to j passing through no intermediate vertex
greater than k-1. This will be the result after k-1 iterations. Hence kth iteration explores
whether k lies on optimal path. A shortest path from i to j passes through no vertex greater
than k, either it goes through k or does not.

If it does, Ak[i,j] = Ak-1[i,k] + Ak-1[k,j]

If it does not, then no intermediate vertex has index greater than k-1. Then Ak[i,j] = Ak-1[i,j]

Combining these two conditions we get

Ak[i,j] = min{ Ak-1[i,j], Ak-1[i,k] + Ak-1[k,j]}, k>=1 (A0[i,j]= cost(i, j))

Consider the directed graph given below.

https://www.freshersnow.com/
Cost adjacency matrix for the graph is as given below

Copy the cost values to the matrix A. So we have A0 as

Matrix A after each iteration is as given below.

Recurrence relation is:

𝑨𝒌 (i, j) = min { 𝑨𝒌−𝟏 (i,j), 𝑨𝒌−𝟏 (i,k) + 𝑨𝒌−𝟏 (k,j) } , k ≥1

Algorithm

Void All Paths ( float cost[ ][ ], float A[ ][ ], int n)

// cost[1: n, 1:n] is the cost adjacency matrix of a graph with

https://www.freshersnow.com/
// n vertices; A[i, j] is the cost of a shortest path from vertex
// i to vertex j. cost[i, i] = 0.0, for 1≤ i≤n
for (int i=1; i<=n; i++)

for (int j=1; j<=n; j++)

A[i][j] = cost[i][j];

for (int k=1; k<=n; k++)

for (i=1; i<=n; i++)

for (j=1; j<=n; j++)

A[i][j] = min{A[i][j], A[i][k] + A[k][j])};

The above is the algorithm to compute lengths of shortest paths.

Time complexity of the algorithm is O (n3).

0/1 KNAPSACK PROBLEM (DYNAMIC KNAPSACK)

The 0/1 knapsack problem is similar to the knapsack problem as in the Greedy method except
that the xi’s are restricted to have a value of either 0 or 1.

We can represent the 0/1 knapsack problem as:

The 0/1 knapsack problem thus is:

Maximize ∑ pi xi

subject to: ∑ wi xi ≤ m(capacity) and xi=0 or 1 where l≤i≤n

The decisions on the xi are made in the order xn,xn-1 …x1. Following a decision on xn, we may
be in one of two possible states the capacity remaining in the knapsack is m and no profit has
accrued or the capacity remaining is m-wn and profit of pn has accrued). It is clear that the
remaining decisions xn-1,…..,x1 must be optimal w.r.t the problem state resulting from the
decision xn. Otherwise xn,…..x1 will not be optimal.

Let fn(m) be the value of optimal solution to KNAP (1,n,m).

Since principle of optimality holds good, we obtain

fn(m)=max{fn-1(m),fn-1(m-wn)+pn}

in general,

fi(y)=max{fn-1(y),fn-1(m-wi)+pi}

A decision on variable xi involves deciding which of values 0 or 1 is to be assigned to it.

https://www.freshersnow.com/
Where Si is the set of all pairs for fi including (0,0) and fi is completely defined by the
pairs (pi,wi)

Si+1 may obtained by merging Si and S1 i. This merge corresponding to taking the make of
two functions fi-1(x) and fi-1(x-wi)+pi in the object function of 0/1 Knapsack problem

So, if one of Si-1 and S1i has a pair (pj, wj) and the other has a pair (pk,wk) and pj ≤ pkwhile
wj≥wk. Then the pair (pj ,wj) is discarded this rule is called purging or dominance rule when
generating Si all the pairs (p,w)with w>m may also be purged

If we try to remove ith item then profit of ith item is reduced from total profit and weight of ith
item is reduced from total weight which belongs to the profit and weight of (i-1)th item

Outline to solve the 0/1 problem

Initially we start with S0= {0, 0} i.e profit =0 & weight=0 (bag is empty)

Addition

Then, S0 1 is obtained by adding P1, W1 to all pairs in S0. For general i, S I 1 is obtained by
adding Pi, Wi to S i-1 The following recurrence relation defines S I 1.

S1 i= {(P, W)/ (P-Pi, W-Wi)  Si}

Merging or union Si = Si-1 U S1i

Purging (Dominance) Rule

Take any two sets Si and Si 1 consisting of pairs(Pj , Wj) and (Pk ,Wk). The purging rule states
that if Pj≤ Pk and Wj ≥ Wk then (Pj, Wj) will be deleted. In other words, remove those pairs
that achieve less profit using up more weight.

problem : m=6 n=3 (W1,W2,W3)= (2,3,3), and (P1,P2,P3)=(1,2,4)

In other words, the given profit, weight pairs are: (1,2) (2,3) and (4,3)

Solution: Initially take S0 ={0,0}

Addition:

S01 = Add (P1, W1) for all pairs in S0

S0 1 = {(1,2)} because S0 is (0,0)

Merging operation:

Si = Si-1 + Si-1 1

S1 = S0 U S0 1 there fore S1 = {(0,0) U{(1,2)} = {(0,0) (1,2)}

S12 = S1 + (P2 , W2)

https://www.freshersnow.com/
= {(0,0),(1,2)}+{2,3)}

S12 = {(2,3),(3,5)}

S2 = S1 U S12 = {(0,0)(1,2)}U{(2,3),(3,5)}

S2 = {(0,0),(1,2),(2,3),(3,5)}

S13 = S2 + (P3, W3)

= {(0,0),(1,2),(2,3),(3,5)}+{(4,3)}

S1 3 = {(4,3),(5,5),(6,6),(7,8)}

S3 = S3-1 U S13 = S 2 U S13

= {(0,0),(1,2),(2,3),(3,5)}U{(4,3),5,5),(6,6),(7,8)}

={(0,0),(1,2),(2,3),(3,5),(4,3),(5,5),(6,6),(7,8)}

Tuple (7,8) is Discarded because the weight (8) exceeds max capacity of Knapsack m=6 so,
S3 ={(0,0),(1,2),(2,3),(3,5),(4,3),(5,5),(6,6)

Applying purging rule:

(Pj,Wj) = (3,5); (Pk,Wk) = (5,3)

Here, (3, 5) will be deleted because 3≤5 and 5≥3

This purging rule has to be applied for every Si. In this example, no purging was necessary in
S1 and S2.

S3 = {(0,0),(1,2),(2,3),(4,3),(5,5),(6,6)}

(2,3) will be deleted because 2≤4 and 3≥3.

S3 = {(0,0),(1,2),(4,3),(5,5),(6,6)}

Finding optimal Solution

The searching process is done starting with the last tuple (P1, W1) of Sn. A set of 0/1 values
for xi ‘s such that ∑ pi xi=P1 and ∑ wi xi=W1 . This can be determined by carrying out a
search through the si s.

If (P1,W1)  Sn-1 , we can set xn=0. Otherwise (if (P1, W1) not in S n-1), we set Xn to 1.
Then, we will continue this search recursively through all Sn-1, Sn-2, …., S0 with either (P1,
W1) in the case xn = 0 or (P1-pn,W1-wn) in the case xn=1.

In the above example, last tuple in S3 is (6,6) but (6, 6) S2. Hence x3=1.

Now, we need to search for (6-P3, 6-W3) which is (2,3).

https://www.freshersnow.com/
(2, 3)  S2 but (2,3) S1 So, x2=1.

Now, we need to search for (2-P2, 3-W2) which is (0,0).

(0,0)  S1 and (0,0)  S0 . Hence, X1=0.

Therefore the optimal sol is {x1=0,x2=1,x3=1} meaning weight 2 and weight 3 are taken and
weight 1 is ignored.

Informal knapsack algorithm

1. Algorithm DKP(p, w, n, m)
2. {
3. S0 := {0,0)};
4. for i :=1to n-1 do
5. {
6. Si-1 :={(P,W)(P-pi,W-wi)  Si-1 and W≤m};
7. Si :=Marge Purge(Si-1, Si-11);
8. }
9. (PX,WX) :=last pair in Sn-1
10. (PY,WY) :=(P’ +pn,W’ +wn) where W’ is the largest W in
11. any pair in S n-1 such that W+ wn≤m;
12. //Trace back for xn,xn-1….,x1.
13. if(PX > PY) then xn :=0;
14. else xn :=1;
15. Trace Back For(xn-1,…..x1);
16. }

TRAVELLING SALESMAN PROBLEM

 Let G = (V,E) be a directed graph with edge cost cij is defined such that cij >0 for all i
and j and cij = ,if (i,j)  E.
Let V =n and assume n>1.

 The traveling salesman problem is to find a tour of minimum cost.

 A tour of G is a directed simple cycle that includes every vertex in V.
 The cost of the tour is the sum of cost of the edges on the tour.
 The tour is the shortest path that starts and ends at the same vertex i.e 1.
 We know that the tour of the simple graph starts and ends at vertex 1. Every tour
consists of an edge <i, k> for some k  V-{1} and a path from k to 1. Path from k to 1
goes through each vertex in V- {1, k} exactly once. If tour is optimal, path from k to 1
muat be shortest k to 1 path going through all vertices in V-{1, k}. Hence the
principle of optimality holds.
Let g(i, S) be length of shortest path starting at i, going through all vertices in S
ending at 1. Function g (1, V-{1}) is the length of optimal salesman tour. From the
principle of optimality

https://www.freshersnow.com/
Use eq. (1) to get g (i, S) for |S| =1. Then find g(i, S) with |S|=2 and so on.

APPLICATION:

1. Suppose we have to route a postal van to pick up mail from the mail boxes located at
‘n’ different sites.
2. An n+1 vertex graph can be used to represent the situation.
3. One vertex represents the post office from which the postal van starts and return.
4. Edge <i,j> is assigned a cost equal to the distance from site ‘i’ to site ‘j’.
5. The route taken by the postal van is a tour and we are finding a tour of minimum
length.
6. Every tour consists of an edge <1,k> for some k  V-{} and a path from vertex k to
vertex 1.
7. The path from vertex k to vertex 1 goes through each vertex in V-{1,k} exactly once.
8. the function which is used to find the path is
g(1,V-{1}) = min{ cij + g(j,S-{j})}

9. g(i,s) be the length of a shortest path starting at vertex i, going

through all vertices in S,and terminating at vertex 1.

10. The function g(1,v-{1}) is the length of an optimal tour.

STEPS TO FIND THE PATH:

1. Find g(i,) =ci1, 1≤ i <n, hence we can use equation(2) to obtain g(i,s) for all s to size
1.
2. That we have to start with s=1,(ie) there will be only one vertex in set ‘s’.
3. Then s=2, and we have to proceed until |s| <n-1.
4. for example consider the graph.

g(i,s) set of nodes/vertex have to visited.

https://www.freshersnow.com/
starting position

From eq g(i,s) =min{cij +g(j,s-{j}) we obtain g(i,S) for all S of size 1,2…

When |s|<n-1, the values of I and S for which g(i,S) is needed such that

i  1, 1 s and i s.

|s| =0

i =1 to n. g(i, )=ci1,1≤ i≤ n
g(1,) = c11 => 0; g(2,) = c21 => 5; g(3,) = c31 => 6; g(4,) = c41 => 8

|s| =1

i =2 to 4

g(2,{3}) = c23 + g(3,) = 9+6 =15

g(2,{4}) = c24 + g(4,) = 10+8 =18

g(3,{2}) = c32 + g(2,) = 13+5 =18

g(3,{4}) = c34 + g(4,) = 12+8 =20

g(4,{2}) = c42 + g(2,) = 8+5 =13

g(4,{3}) = c43 + g(3,) = 9+6 =15

| s| =2

i  1, 1 s and i s.

g(2,{3,4}) = min{c23+g(3{4}),c24+g(4,{3})}

= min{9+20,10+15} = min{29,25} =25

g(3,{2,4}) =min{c32+g(2{4}),c34+g(4,{2})}

= min{13+18,12+13} = min{31,25} = 25

g(4,{2,3}) = min{c42+g(2{3}),c43+g(3,{2})}

= min{8+15,9+18} = min{23,27} =23

|s| = 3

from eq(1) we obtain

https://www.freshersnow.com/
g(1,{2,3,4})=min{c12+g(2{3,4}),c13+g(3,{2,4}),c14+g(4,{2,3})}

= min{10+25,15+25,20+23} = min{35,35,43} =35

optimal cost is 35

the shortest path is,

g(1,{2,3,4}) = c12 + g(2,{3,4}) => 1->2

g(2,{3,4}) = c24 + g(4,{3}) => 1->2->4

g(4,{3}) = c43 + g(3{}) => 1->2->4->3->1

so, the optimal tour is 1  2  4 3  1

Optimal Binary Search Tree

Consider a fixed set of words and their probabilities. The problem is to arrange these
words in a binary search tree

Let us consider S is the set of words

S= {if, for, int, while, do}

if the following are Two Binary Search Trees (B S T ) For given set S. here assumption is
each word has same probability no unsuccessful search

for
for

do int
do
whil
e
if whil
ee
int

if
(a) (b)

We require 4 comparisons to find out an identifier in worst case in fig(a).But in fig (b) only 3
comparisons on avg the two trees need 12/5 ,11/5 comparisons

https://www.freshersnow.com/
In General, we can consider different words with different frequencies (probabilities) and un
successful searches also

Let Given set of words are {a1, a2, a3, …….an} With a1< a2 <….. an

Let P(i) be the probability for searching of ai

q(i) be the probability for unsuccessful search so clearly

∑p(i) + ∑q(i) =1

1≤i≤n 0≤i≤n

So Let us construct optimal Bin search true

To obtain a cost function for BST add external nodes in the place of every empty sub
true. These nodes called external nodes.

for
for

do int
do wh
ile

int
if wh
ile

If BST represents ‘n’ identifiers then there will be exately ‘n ‘ internal nodes and n+ 1
external nodes every internal node represent a point where successful search may terminate
and every external node represents a point where an unsuccessful search may terminates

If a successful search terminates at an internal node at level L, then L interations are required.
So expected cost for the internal node for ai is P(i) * Level (ai)

For unsuccessful search terminate at external node the words which are not in BST can be
partioned into n+1 Equalnce classes. Equvalence class E0 represents least among all the
identifers in BST& The equalence classes ‘En’ represents greatest among all Elements in BST
Hence search terment at level Ei-1

∑p(i)level (ai) + ∑q(i) (level (Ei)-1)

1≤i≤n 0≤i≤n

we can obtain optimal BST for the above value is min

https://www.freshersnow.com/
problem:

Let P(i)=q(i)=1/7, for all i in the BST The following word set(a1,a2,a3)= (do,it,while)

do
Whil if
e

if do Whil if
e
Whil
do e
(a) (b) (c)

do
While

do While

if
if

(d) (e)

with Equal probability p(i) = q(i)=1/7 for all i

we have cost (a) = 1/7* 1+1/72+1/73+1/72+1/72+1/7*1

= 15/7

Cost (b) = 1/71+1/72+1/72+1/72+1/72+1/72+1/7*2

= 13/7

Cost (c) =cost(d)=cost(e)=15/7

Tree b is optimal

with p(1) = 0.5,p(2) = 0.1,p(3) = 0.05,q(0)=0.05,q(1)=0.1,q(2)=0.05,q(3)=0.05

cost(Tree a)=2.05

https://www.freshersnow.com/
cost (Tree c)= 1*0.5 + 2*0.1 + +3*0.05+3*0.15+3*0.11+2*0.05 +1 * 0.05

= 0.5+0.2 + 0.15 + 0.45 + 0.3 + 0.10 + 0.05

=1.75

To apply dynamic programming to the problem of obtaining an optimal BST , we need to

construct the tree as the result of a sequence of decisions and observe the principal of
optimality

One possible approach to this would be to make a desition as to which of the

ai’s should be assigned to the root node of the tree. If we choose ak then the internal nodes
for a1,a2,………ak-1 will lie one left sub tree of the root. The remaining will be in the right
sub tree r.

ak a2

a1 a3

a1 …….ak-1 ak+1 ….. an E0 E1 E2 E3

E0…….Ek-1 Ek ………..En

Cost (l) = ∑ p(i) * level (ai) + ∑q(i)* level (E i-1)

1≤i≤k-1 0≤i≤k-1

Cost (r) = ∑ P(i) * level (ai) + ∑ q(i) *(level (Ei)-1)

k+1 ≤ i≤n k<i≤n

By using the forumula

w (i ,j) = q(i) + ∑ j( (q(l) + p(l)) we obtain

l=i+1

the following as the expected cost of search tree

p(k) +cost (l) + cost (r) + w(0,k-1) + w (k , n)

For left sub tree cost is c (0,k-1) and using (0,k-1)

For right sub Tree cost is c(k,n) and using w(k,n)

cost of the tree is

https://www.freshersnow.com/
c(0,n) = min∑ { c (0,k-1) + c(k,n) + w(0,k-1) + w(k,n) + p(k) }

1≤k≤n Total true weight

= min ∑ { (0,k-1) + c (k,n) + w (0,n) }

1≤k≤n

In General,

c(i,j) = min ∑{ c (i,k-1) + c(k,j) + w(i,j) + w(i,j) }

i≤k≤j

The above Eq can be solved for c (0,n) by first computing all c(i,j) such that j-1 = 1

Next we compute all c(i,j) of such that j-i=2 then c(i,j) with j-i=3 and so on.we record
all the values of r(i,j) of each tree then optimal BST can be computed from these r (i,j)

Note: Initial values are c (i,i) = 0

w(i,i) = q(i)

r(i,i) = 0 for all 0≤i<4

From observation, w(i,j) = p( j) + q( j) + w (i,j-1)

(Problem and Solution discussed in the class room)

MULTI SATAGE GRAPHS

https://www.freshersnow.com/
VERTEX 1 2 3 4 5 6 7 8 9 10 11 12

COST 16 17 9 18 15 7 5 7 4 2 5 0
DISTANCE 2/3 7 6 8 8 10 10 10 12 12 12 12

https://www.freshersnow.com/
Backtracking
• Problem which deals with searching for a set of Solutions or which ask for an Optimal Solution
satisfying Some Constraints can be solved using Backtracking.
• The desired solution is expressible as an N-Tuple (X1, X2, _ _ _ _ _ _, Xn) where Xi are chosen from
some Finite Set Si. The problem to be solved calls for finding one vector that maximizes or
minimizes a criterion function P(X1 , X2, _ _ _ _ _ _, Xn) sometimes it seeks all vectors that satisfies
P.
• Let mi is the size of set Si then there are m = m1 . m2 . m3 . ………. mn n – Tuples that are possible
candidate sets.
• Brute force approach evaluates all possible m-Tuples
• Backtracking yields same problem with less than m trails
o The basic idea of Backtracking is to build up the Solution vector one component at a
time and
o to use modified criterion function Pi(X1, X2, _ _ _ _ _ _, Xi)(Bounding Function) to test
whether the vector being formed has any chance of success.
o The major advantage of this method is, If it is realized that the partial vector (X1, X2, _ _ _
_ _ _, Xi) can in no way lead to an optimal solution, then Mi+1, _ _ _ _ Mn possible vectors
can be ignored entirely.
• Many Problems solved using Backtracking require that all the solutions satisfy some complex set
of Constraints
o These are classified as
§ Explicit
§ Implicit
o Explicit constraints are rules that restrict each Xi to take on values only from a given set
o Implicit Constraints describe the way in which the Xi must relate to each other
Examples
In 8-Queens Problem the solution space contains 88 tuples 8-Tuples
Explicit Constraints
Si = {1,2,3,4,5,6,7,8}
Implicit Constraints
No two Xi’s can be the same (All Queens are in different Columns)
No two queens can be on the same Diagonal
• Solution Space is facilitated by Tree Organization called State Space Tree

https://www.freshersnow.com/
State Space tree for the 4-Queens Problem
1

3 8

4 6 9

5 7 1
0

Give Numbers to nodes in the above tree using DFS

State Space tree

Searching of the State Space tree for the solution (vector) can be performed as Begin with the root
node and generate other nodes.
Live Node: A node which has been generated and all of whose children have not yet been
generated is called Live Node
E-Node: The Live node whose children are currently being generated is called E-node
Dead Node: Dead node is a generated which is not to be expanded further or all of whose children
have been generated

Method 1:
As soon as a new child “C” of the current E Node “R” is generated, this child will become the
new E-Node. Then R will become E-node again when the sub-tree C has been Fully Explored.
(This is the DFS Strategy)
Method 2:
The E-Node will remain E-node until it is Dead.

In the both of the methods, Bounding Functions are used to kill Live Nodes with out generating all of
their Children.
Depth First Generation (Method 1) with Bounding Function is called Backtracking.

https://www.freshersnow.com/
Example of a backtrack solution to the 4-queens problem

Portion of the tree of State Space Tree for 4-Queens problem that is generated during back-tracking
1. Algorithm Backtrack(k)
2. // This schema describes the backtracking process using
3. // recursion. On entering, the first k-1 values
4. //x[1]x[2],….x[k-1] of the solution vector
5. //x [1: n] have been assigned. x [] and n are global.
6. {
7. for (each x[k] €T(x[1],…..,x[k-1])do
8. {
9. if (Bk(x[1],x[2],….,x[k])≠0)then
10. {
11. if(x[1],x[2],….,x[k] is a path to an answer node)
12. then write (x[1:k]);
13. if (k<n) then Backtrack(k+1);
14. }
15. }
16. }

https://www.freshersnow.com/
1. Algorithm IBacktrack(n)
2. // This schema describes the backtracking process.
3. // All solutions are generated in x{1: n} and printed
4. // as soon as they are determined.
5. {
6. k := 1;
7. while (k≠ 0) do
8. {
9. if (there remains an untried x[k] €T (x[1],x[2],…..,
10. x[k-1] and Bk(x[1],….,x[k])is true)then
11. {
12. if(x[1],….,x[k] is a path to an answer node)
13. then write(x[1:k]);
14. k :=k+1; // Consider the next set.
15. }
16. else k:= k-1;// Backtrack to the previous set.
17. }
18. }

8- Queens of N-queens problem:

• We try to find all ways to place “n” not attacking queens in a chess board.
• We represent solution as N-Tuple (X1, X2, _ _ _ _ _ _, Xn), where Xi is the column of ith row
where the ith queen is placed.
• All Xi’s are distinct (not in the Same Column) So our Solution Space is reduced from nn to n!
n-Tuples.
• How to Place not in the Same Diagonal is crucial.
• Dsfsd

In the above board the Diagonal (3,1) and (7,5) is normal Diagonal
Where as (1,7) and (4,4) is opposite Diagonal

In normal Diagonals Difference Between Row and Column is Same

In Opposite Diagonals Sum and Row and Columns is Same
By Keeping these things in mind we formulate a rule that
If (I,j) and (k,l) are two cells then
In Normal Diagonals i-j = k-l

https://www.freshersnow.com/
In Opposite Diagonals i+j = k+l

Which implies |j-l| = |i-k|

• With the help of this equation we Frame the Function Place()

1. Algorithm place(k,i)
2. // Returns true if a queen can be placed in kth row and
3. //ith column. Otherwise it returns false. x[]is a
4. //global array whose first (k-1) values have been set.
5. //Abs(r) returns the absolute value of r.
6. {
7. for j :=1 to k-1 do
8. if((x[j] =i) // Two in the same column
9. or (Abs(x[j]-i) = Abs(j-k)))
10. // or in the same diagonal
11. then return false;
12. return true;
13. }

Algortihm N-Queen is developed using the concept of Backtracking

1. Algorithm N Queen(k,n)
2. // Using backtracking, this procedure prints all
3. //possible placement of n queens on an n x n
4. //chessboard so that they are nonattacking.
5. {
6. for i:=1 to n do
7. {
8. if place(k, i) then
9. {
10. x[k] := i;
11. if (k = n] then write (x[1 ; n]);
12. else NQueens (k+1,n);
13. }
14. }
15. }

https://www.freshersnow.com/
Sum of subsets problem:

• Suppose we are given n Distinct Positive numbers (Called Weights) and we desire to find all
the combinations of these numbers whose sums are m. This is called Sum of Sub Sets
Problem.
o Ex: n=4, (w1, w2, w3, w4) = (11, 13, 24, 7) & m = 31 then the possible subsets are
(11, 13, 7) & (24, 7)
• We could formulate this problem using either Fixed or variable sized Tuple
• In Fixed Sized Tuple representation the solution subset is represented by n-Tuple where the
elements in this solution vector are either 0 or 1. For Above example in fixed size Tuple is
(1,1,0,1) or (0,0,1,1)
• In Variable Size representation we represent our solution set with the index of the element
in the actual set
o For our above example our solution subsets are (1,2,4) or (3,4)
• We Consider the Backtrack solution using fixed Size representation.

• Consider a node at depth i

• weightSoFar (i.e. s) = weight of node, i.e., sum of numbers included in partial solution node
represents
• totalPossibleLeft (i.e. r) = weight of the remaining items i+1 to n (for a node at depth i)
• A node at depth i is non-promising
if (weightSoFar + totalPossibleLeft < S )
or (weightSoFar + w[i+1] > S )
• To be able to use this “promising function” the wi must be sorted in non-decreasing order

https://www.freshersnow.com/
1. Algorithm SumOfSub(s, k ,r)
2. //Find all subsets of w[1 :n]that sum to m. The values of x[j],
3. //1≤j<k, have already been determined. s = ∑ k-1 w[j]* x[j]
j=1
4. //and r=∑nj=k w[j]’s are in nondecreasing order. n
5. //It is assumed that w[1]≤ m and=∑ni=1 w[i] ≥m
6. {
7. // Generate left child. Note: s+w[k]≤m since Bk-1 is true.
8. x[k]:=1
9. if (s+w[k]=m) then write (1:k); // Subset found
10. // There is no recursive call here as w[j] > 0,1≤j≤n.
11. else if (s+w[k]+w[k+1]≤m)
12. then Sum Of Sub(s+w[k],k+1,r-w[k]);
13. //Generate right child and evaluate Bk
14. if((s+r-w[k]≥m) and (s+w[k+1]≤m)) then
15. {
16. x[k]:=0;
17. Sum Of Sub(s, k+1,r-w[k]);
18. }
19. }
Recursive backtracking algorithm for sum of subset problem

Graph Coloring
• Let G be a Graph and m be a +ve integer
• Nodes of G must be colored with ‘m’ Colors such that no two adjacent nodes have the same
color yet only ‘m’ colors are used.
• This is termed “m-color ability Decision” Problem
• This Problem asks for the smallest integer m for which the graph G can be colored
• This Integer is referred to as the Chromatic number of the Graph
• Ex:
• For a Planar Graph the maximum number for the chromatic number is 4.
1. Algorithm mColoring(k)
2. // This algorithm was formed using the recursive backtracking
3. // schema. The graph is represented by its Boolean adjacency
4. // matrix G[1 :n, 1:n]. All assignments of 1,2,…, m to the
5. // vertices of the graph such that adjacent vertices are
6. // assigned distinct integers are printed. k is the index
7. // of the next vertex to color.
8. {
9. repeat
10. {// generate all legal assignments for x[k].
11. Next value (k);// Assign to x(k) a legal color.
12. if (x[k]=0) then return; // No new color possible
13. if(k=n) then // At most m colors have been
14. // used to color the n vertices.
15. write (x[k+1];
16. else mColoring(k+1)
17. } until ( false);
18. }
Finding all – m coloring of a graph

https://www.freshersnow.com/
• The NextValue(k) will assign the next possible color to the vertex k. This function will checks
the newly assigned color is not as the color of the adjacent vertex(which were already
assigned)
• mColoring(k) calls NextValue(k) and checks whether all vertices were colored(k=n or not), if
not it will calls mColoring(k+1)

1. Algorithm Next Value(k)

2. // x[1],…..,x[k-1] have been assigned integer values in
3. // the range [1,m] such that adjacent vertices have distinct
4. //integers A value for x[k] is determined in the range
5. //[o,m].x[k] is assigned the next highest. numbered color
6. //while maintaining distinctness from the adjacent vertices
7. //of vertex k. If no such color exists, then x[k] is 0
8. {
9. repeat
10. {
11. x[k]:=(x[k]+1) mod (m+1); // Next highest color.
12. if ((x[k]=0) then return; // All colors have been used.
13. for j:= 1 to n do
14. { // Check if this color is
15. // distinct from adjacent colors.
16. if((G[k,j]≠0) and (x[k]=x[j]))
17. //if (k,j) is and edge and if adj.
18. //vertices have the same color.
19. then break
20. }
21. if (j=n+1) then return; // New color found
22. } until (false); //Otherwise try to find another color.
23. }

Generating a next color

Hamiltonian Cycle
• Let graph G=(V,E) be a connected graph with n vertices. Hamiltonian Cycle is a round trip
path along n edges of G that visits every vertex once and returns to its starting position.
• The backtracking solution vector(x1,x2 ……. xn) is defined so that xi represents the ith visited
vertex of the proposed cycle. Now all we need is determine how to compute the set of
possible vertices for xk if x1,x2 ……. xk-1 have already been chosen. If k =1 then x1 can be any
of the n vertices.
• To avoid printing the same cycle n times, we require that x1 =1.
• If 1<k<n, then xk can be any vertex v that is distinct from x1, x2 ……. xk-1 and v is connected by
an edge to xk-1. The vertex Xn can only be the one remaining vertex and it must be connected
to both xn-1 and x1.

https://www.freshersnow.com/
1. Algorithm Hamiltonian(k)
2. // This algorithm uses the recursive formulation of
3. // backtracking to find all the Hamiltonian cycles
4. //of a graph. The graph is stored as an adjacency
5. //matrix G[1: n,1:n]. All cycles begin at node 1.
6. {
7. repeat
8. { // Generate values for x[k]
9. Next value(k);// Assign a legal next value to x[k]
10. if(x[k]=0) then return;
11. if (k=n) then write (x[1:n]);
12. else Hamiltonian(k+1);
13. } until (false);
14. }

1. Algorithm Next values(k)

2. // x[1:k-1]is a path of k-1 distinct vertices. If[k]=0,then
3. // no vertex has as yet been assigned to x[k]. After execution
4. //x[k]is assigned to the next highest numbered vertex which
5. //does not already appear in x[1:k-1]and is connected by
6. // an edge to x[k-1].Otherwise x[k]=0.If k=n, then
7. // in addition x[k] is connected to x[1]
8. {
9. repeat
10. {
11. x[k]:=(x[k]+1)mod (n+1); //Next vertex
12. if(x[k]=0) then return;
13. if(G[x[k-1],x[k]≠0) Then
14. {// Is there an edge?
15. for j:=1 to k-1 do if (x[j]-x[k] then break;
16. //check for distinctness.
17. if (j=k) then // if true, then the vertex is distinct.
18. if((k<n) or((k=n) and G[x[n],x[1]]≠0))
19. then return;
20. }
21. } until (false)
22. }

Generating a next vertex

https://www.freshersnow.com/
Basic concepts
Decision Problems:

 When a problem is given, the first thing concern is:

 The problem has a solution or not?

Example-1:

o Consider graph G = (V, E), find if there is a path from vertex ‘X’ to vertex ‘Y’ whose
length is at most 10.

o Suppose a number is given to you, say 1111111111111111111111111 (25 one’s), and

asks you find it is prime or not.

Observation:

 The above example problems are called decision problems, because the problem has
solution (answer), i.e. is either YES or NO.

 Some decision problems can be solved efficiently, using time polynomial to the size of
the input of that problem

 Generally ‘P’ is used to denote the set of all these polynomial-time solvable problems

Example:

 Find the shortest path from vertex ‘X’ to vertex ‘Y’ for a graph G.

o First apply Dijkstra's algorithm and then give the correct answer

o Hence, this problem is in ‘P’

 One more categorization of decision problems is to observe if the problem can be verified
in time polynomial toward the input size.

Example:

 For the same graph G, if there is a path from ‘X’ to ‘Y’ with length is less than 10, then:

1. Find the sequence of vertices without repetition in any path from ‘X’ to ‘Y’ with
length is less than 10.

2. Verify that the resultant path is in poly-time.

 This type of problem is polynomial-time verifiable.

https://www.freshersnow.com/
Examples:

1. Given a graph G (V, E) does the graph contain a Hamiltonian path?

2. Is a given integer ‘x’ a composite number?

3. Given a set of numbers, can be divide them into two groups such that their sum is the
same (equal)?

The class NP:

 For some problems, directly we cannot provide solution in polynomial time. Such
problems can be represented by ‘NP’.

 A problem is said to be Non-deterministically Polynomial (NP) if we can find a non-

determinsitic Turing machine that can solve the problem in a polynomial number of
nondeterministic moves.

o ‘N’ is non-deterministic guessing power of computer

o ‘P’ is polynomial-time

Examples:

Polynomial problems: searching, sorting and string editing – these problems need the
time O(log n), O(n log n) and O(mn) respectively.

Non-Polynomial problems: travelling sales person problem and Knapsack problem --

these problems need time O(n22n) and O(2n/2) respectively.

The two classes of NP problems are:

1. NP- complete problems

2. NP- hard problems

NP-complete problems:

 A decision problem E is NP-complete if and only if every problem in the class NP is

polynomial-time reducible to E, i.e.NP-complete can be solved in polynomial time.

Example:

 The decision versions of the Travelling sales person, graph coloring problem and
Hamiltonian cycle problem are NP-complete problems.

NP- hard problems:

https://www.freshersnow.com/
 Optimization problems whose decision versions are NP-complete are called NP-hard.

 All NP-complete problems are NP-hard but some NP-hard problems are not known to be
NP-complete.

Deterministic algorithm:

The class ‘P’ consists of all problems that can be solved in polynomial time, O(Nk), by
deterministic computers.

A deterministic algorithm generates the same output for a problem instance every time
the algorithm is run

Example: Searching algorithm

/input: an array A[1..N] and an element ‘x’ that is in the array

//output: the position of the element in array A

search(element, A, i)

if (A[i] == x)

then return i;

else

return search(x, A, i+1);

Let us say, input A = {7, 5, 3, 8, 9, 3} and x = 9 then the search begins from 7, then 5,
then 3 and so on until a successful search or element not found.

The complexity of the algorithm is O(n) since the input is not sorted.

Non-deterministic computer:

 A nondeterministic computer has more than one output state and it “chooses” the correct
one, is known as “nondeterministic choice”.

 A nondeterministic algorithm result is not essentially unique.

 A non-deterministic algorithm is a two-stage procedure that takes as its input an instance

of a decision problem and does the following:

https://www.freshersnow.com/
o Stage 1: Non-deterministic (“Guessing”) Stage: An arbitrary string ‘S’ is
generated that can be thought of as a candidate solution to the given instance ‘x’.

o Stage 2: Deterministic (“Verification or checking”) Stage: A deterministic

algorithm takes both ‘x’ and ‘S’ as its input, and results ‘yes’ if ‘S’ represents a
solution to instance ‘x’. If the checking stage of a nondeterministic algorithm is of
polynomial time-complexity, then this algorithm is called an NP
(nondeterministic polynomial) algorithm

It makes use of additional functions:

• choice(S): arbitrarily chooses one of the elements of the set S

• failure(): signifies unsuccessful computation

• success(): signifies successful computation

 A nondeterministic algorithm makes an order of choices for successful termination.

 Two-stage procedure for problem A is:

o Stage-I: a method makes a guess about the possible solution for problem A.

o Stage-II: a method checks if the guessed solution is really a solution for problem
A.

Example-1: Search algorithm

STAGE-1:

NonDeterministicSearch(x, A)

//input: an array A[1..N] and an element ‘x’ that is in the array

//output: the position of the element in array A

// informally calls an NDS function, which makes the right choice

https://www.freshersnow.com/
i := NDS(x, A, 1); // calling function NDS()

return i;

The complexity of the algorithm is O(1).

STAGE-2:

NDS(x, A, i)

if (A[i] == x) then

return i;

else

return NDS(x, A, i+1);

Example-2:

Nondeterministic algorithm to sort the given element of an array A[1..n] is as shown below:

Algorithm NDSORT(A, B)

{ // A and B are arrays with size ‘n’

for i := 1 to n do

j := choice(1 . . . n); //selecting correct nondeterministic choice

if (B[j]!= 0) then

failure();

B[j] := A[i]; // coping element of A to B

endfor

for i := 1 to n−1 do //verifying the order of elements

if (B[i] > B[i + 1]) then

failure(); // if the elements are not in order

https://www.freshersnow.com/
endfor

write(B);

success();

Note:

 We consider only nondeterministic decision algorithms whose output is success or

failure.

 Many optimization problems can be stated as decision problems with the property that:

o The decision problem can be solved in polynomial time if and only if the
optimization problem can be solved in polynomial time.

Time Complexity of Nondeterministic Algorithm:

 Time complexity of a nondeterministic algorithm is O(f(n)), for all input size ‘n’ that
result in successful completion and the time required is at most cf(n) where ‘c’ and ‘n0’
are positive constants

For failure or not successful completion the time required is O(1)

Example-1:

Nondeterministic algorithm for Knapsack problem:

Algorithm NDKnapsack( p, w, n, m, r, x)

// p-profit, w-weight, n-input size, m-knapsack capacity, r-least profit, x-object

// initialize W and P to zero

for i := 1 to n do

x[i] := choice(0, 1); // selecting an object (choice of nondeterministic)

if ( (∑ wixi > m) || (∑ pixi < r) ) then // knapsack(bag) is empty(not full) or No

profit

failure();

else

https://www.freshersnow.com/
success();

The time complexity of this algorithm appears to be O(n)+O(m), where, ‘m’ is the length of
the input.

Hence, the algorithm’s time complexity is polynomial in the size of the input.

Example-2:

Nondeterministic algorithm for Satisfiability (SAT):

 Let x1, x2, … , xn be boolean variables and denotes the negation of xi

 A formula is an expression of literals combined with boolean operations AND( ) and OR

( ) , where a literal is either a variable or its negation

 A formula is in conjunctive normal form (CNF) if it is of the form ci where, clause ci = lij

 A formula is in disjunctive normal form (DNF) if it is of the form ci where, clause ci = lij

 For example, (x3 ) (x1 ) is in CNF, then the DNF is (x1 x2) (x3 )

The satisfiability (SAT) problem is to search out whether or not a formula is true for a few
assignment of truth values to the variables.

Nondeterministic Algorithm for SAT:

Algorithm NDSAT(E, n)

//Determine whether the propositional formula ‘E’ is satisfiable

for i := 1 to n do // choose a truth value assignment

xi = Choice (true, false);

if E(x1, x2, … , xn) then

success();

else

failure();

https://www.freshersnow.com/
}

The time of the algorithm is O(n) + time to deterministically evaluate ‘E’.

Therefore, time complexity is O(n) + O(|E|).

Thus, this is a polynomial time nondeterministic algorithm.

NP HARD and NP Complete classes

 P is set of problems which will be solved in polynomial time.

 NP (nondeterministic polynomial time) is the set of problems which will be solved in

polynomial time.

 NP-complete problems are the “hardest” problems in NP, if any NP-complete problem
can be solved in polynomial time, then all NP-complete problems can be solved in
polynomial time and in fact every problem in NP can be solved in polynomial.

Reducibility:

If P is polynomial-time reducible to Q, we denote this P p Q, i.e. a problem P can be reduced to

another problem Q if any instance of P can be “rephrased” as an instance of Q, the solution to
which provides a solution to the instance of P.

NP-Hard problems:

 If all problems R NP are reducible to P, then P is NP-Hard.

NP-Complete problems:

 If P is NP-Hard and P NP then P is NP-Complete.

https://www.freshersnow.com/
NP-Hard problems are not NP-complete.

Example: HALTING problem:

The decision problem HALTING returns TRUE, if, for a given input ‘I’ and a given
deterministic algorithm ‘A’; the algorithm ‘A’ terminates, otherwise it loops forever. There is no
algorithm of any type to solve HALTING.

HALTING problem is NP-hard: (hint: Reduce an instance of Circuit-satisfiability to an

instance of HALTING)

 The reducing algorithm constructs an instance of HALTING from an instance of

CIRCUIT-SAT as follows:

o Construct an algorithm ‘A’ whose input is the encoding of a boolean circuit ‘B’

o ‘A’ tries all possible (2n, where ‘n’ is the number of input lines to the circuit)
combinations to simulate the circuit, and looks to see if the output is 1 for any of
the combinations. If it is, then ‘A’ halts. If not, then ‘A’ goes into an infinite loop

 It is clear from the construction that ‘A’ halts exactly when ‘B’ is satisfiable, and loops
forever exactly when ‘B’ is not satisfiable.

 In addition, we can perform the construction above in polynomial time:

https://www.freshersnow.com/
o The code in algorithm A for the step to generate the 2n combinations is O(n) long;
and if you assume that the number of gates in the boolean circuit is some constant
‘k’ times the number of inputs, then the code to simulate those gates is also O(n)
long

 Note: We only need to consider the length of the code in algorithm ‘A’ and not it’s
running time, in order to have polynomial time reducibility.

Thus, we conclude by definition of NP-hardness that HALTING is NP-hard

HALTING is not NP-complete:

We know that the HALTING problem is undecidable, i.e., there is no algorithm to solve
HALTING. By using this fact, we can prove by contradiction that HALTING is not in NP.

 Let us say a polynomial time verification algorithm for HALTING, given an input string
‘x’ and an algorithm ‘AHNP’ such that A(x, y) = 1, with ‘x’ being an encoding of an
instance of HALTING that returns TRUE and ‘y’ being a certificate that is polynomial
length in ‘x’

 Given such an AHNP(x, y), we can look at the input string xhalt whenever AHNP(xhalt,
y)=1 and conclude that xhalt includes the representation of an algorithm Ahalt that will halt

The algorithm AHNP can be modified to recognize encodings of all algorithms ‘A’ that do
terminate. But this contradicts the assumption that HALTING is undecidable.

Thus, it is not possible to construct a verification algorithm for HALTING.

Therefore, HALTING is not in NP and by definition of NP-completeness, we conclude that

HALTING is not NP-complete.

Hence, NP-Hard problems are not NP-complete.

Cook’s theorem
 The Cook's theorem (Cook–Levin theorem) states that the boolean satisfiability
problem is NP-complete, i.e., any problem in NP can be reduced in polynomial time by
a deterministic Turing machine.

 An important consequence of the theorem is that if there exists a deterministic

polynomial time algorithm for solving boolean satisfiability, then there exists a
deterministic polynomial time algorithm for solving all problems in NP. Crucially, the
same follows for any NP complete problem.

Proof:

https://www.freshersnow.com/
Let us say a NP decision problem D. And polynomial function P and a Turing machine M for
given any instance I (of length ‘n’) of D, together with a candidate certificate c, will check in
time no greater than P(n).

 Let us consider the machine M with q states (0, 1, 2, .., q-1) and a tape alphabet (a1, a2, ..,
as)

 Initially, the tape is inscribed with the problem instance on the squares 1, 2, .., n and the
putative certificate on the squares –m, .., -2, -1

 Square 0 can be assumed to contain a designated separator symbol

 Assume that, the machine halts scanning square 0, and that the symbol in this square at
that stage will be a1 if and only if the candidate certificate is a true certificate

 Here, m <= P(n); since with a problem instance of length ‘n’ the computation is
completed in at most P(n) steps; during this process, the Turing machine head cannot
move more than P(n) steps to the left of its starting point

We define some atomic propositions with their proposed interpretations as follows:

1. For i = 0, 1, .., P(n) and j = 0, 1, 2, .., q-1 the proposition Qij says that after ‘i‘
computation steps, M is in state ‘j’.

2. For i = 0, 1, .., P(n), j = -P(n), …, P(n) and k =1, 2, ..,s; the proposition Sijk says that
after ‘i’ computation steps, square ‘j’ of the tape contains the symbol ak.

3. For i = 0, 1, .., P(n), j = -P(n), …, P(n), the proposition Tij says that after ‘i’ computation
steps, the machine M is scanning square ‘j’ of the tape.

Next, we define some clauses to describe the computation executed by M:

1. At each computation step, M is in at least one state. For each i = 0, 1, .., P(n) we have
the clause Qi0 V Qi1 V … Qi(q-1) giving (P(n)+1)q = O(P(n)) literals altogether.

2. At each computation step, M is in at most one state. For each i = 0, 1, .., P(n) and for
each pair j, k of distinct states, we have the clause (Qij ^ Qik), giving a total of q*(q-
1)*(P(n) + 1) = O(P(n)) literals altogether.

3. At each step, each tape square contains at least one alphabet symbol. For each i = 0, 1, ..,
P(n), and -P(n) j P(n) we have the clause Sij1 v Sij2 v … Sijs; giving
(P(n)+1)*(2P(n)+1)*s = O(P(n)2) literals altogether.

4. In every step, every tape square has at most one alphabet symbol. For every i = 0, 1, ..,
P(n), and -P(n) j P(n), and every distinct pair ak,al of symbols we have the clause (Sijk ^
Sijl); giving a total of (P(n)+1)*(2P(n)+1)*s*(s-1) = O(P(n)2) literals altogether.

https://www.freshersnow.com/
5. In every step, the tape is scanning at least one square. For each i = 0, 1, .., P(n), we have
the clause Ti(-P(n)) v Ti(1-P(n)) v … Ti(P(n)-1) v TiP(n); giving (P(n)+1)*(2P(n)+1)=O(P(n)2)
literals altogether.

6. In every step, the tape is scanning at most one square. For each i = 0, 1, .., P(n) and each
distinct pair j, k of tape squares from -P(n) to P(n), we have the clause (Tij ^ Tik); giving a
total of 2P(n)*(2P(n)+1)*(P(n)+1)=O(P(n)3) literals.

7. Initially, the machine is in state 1 scanning square 1. This is expressed by the two clauses
Q01 and T01 giving just two literals.

8. After the first step, at every step the configuration is determined the preceding step by the
functions T, U, and D. For every i = 0, 1, .., P(n), -P(n) j P(n), k = 0, 1, .., q-1 and l = 1,
2, .., s we have the clauses

Tij ^ Qik ^ Sijl  Q(i+1)T(k, l)

Tij ^ Qik ^ Sijl  S(i+1)jU(k, l)

Tij ^ Qik ^ Sijl  T(i+1)(j+D(k, l))

Sijk  Tij v S(i+1)jk

The fourth of these clauses ensures that the contents of any tape square ther

than the currently scanned square remains the same.

These clauses contribute a total of (12s + 3)(P(n) + 1)(2P(n) + 1)q = O(P(n)2) literals.

9. Initially, the string ai1 ai2 … ain defining the problem instance ‘I’ is inscribed on squares
1, 2, …, n of the tape. This is expressed by the ‘n’ clauses S01i1, S02i2, …, S0nin a total of ‘n’
literals.

10. By the P(n)th step, the machine has reached the halt state, and is then scanning square0,
which contains the symbol a1. This is expressed by the three clauses QP(n)0, SP(n)01 and
TP(n)0 giving another 3 literals.

Altogether the number of literals involved in these clauses is O(P(n)3). Thus, the procedure for
setting up these clauses, given the original machine M and the instance ‘I’ of problem D, can be
accomplished in polynomial time.

We must now show that we have succeeded in converting the problem D into SAT.

 Suppose first that ‘I’ is a positive instance of D

https://www.freshersnow.com/
 This means that there is a certificate ‘c’ such that when M is run with inputs ‘c’ and ‘I’ it
will halt scanning symbol a1 on square0

 This means that there is some sequence of symbols that can be placed initially on squares
-P(n), -P(n-1), .., -1 of the tape so that all the clauses above are satisfied

Hence, those clauses constitute a positive instance of SAT.

Conversely, suppose ‘I’ is a negative instance of D. In that case there is no certificate for ‘I’,
which means that whatever symbols are placed on squares -P(n), -P(n-1), .., -1 of the tape, when
the computation halts the machine will not be scanning a1 on square0. This means that the set of
clauses above is not satisfiable, and hence constitutes a negative instance of SAT.

 Thus from the instance ‘I’ of problem D is constructed in polynomial time, a set of
clauses which constitute a positive instance of SAT if and only ‘I’ is a positive instance
of D.

 Hence, the problem D converted into SAT in polynomial time.

 And since D was an arbitrary NP problem it follows that any NP problem can be
converted to SAT in polynomial time.

https://www.freshersnow.com/

UNIT I Data Structures OU CSE First Year
No ratings yet
UNIT I Data Structures OU CSE First Year
19 pages
Daa All Units Materials
No ratings yet
Daa All Units Materials
77 pages
Daa R20 Unit 1
No ratings yet
Daa R20 Unit 1
21 pages
Introduction To Algorithms: Process of Translating A Problem Into An Algorithm
No ratings yet
Introduction To Algorithms: Process of Translating A Problem Into An Algorithm
17 pages
Group Assignment - Theory of Algorithms
No ratings yet
Group Assignment - Theory of Algorithms
11 pages
Daa
No ratings yet
Daa
137 pages
Final Algorithm Lab Manual
No ratings yet
Final Algorithm Lab Manual
38 pages
Pseudocode Notes
No ratings yet
Pseudocode Notes
9 pages
Data Structures and Algorithms
No ratings yet
Data Structures and Algorithms
45 pages
Ques1: Write An Algorithm To Find The Largest Value Among N 1 Numbers
No ratings yet
Ques1: Write An Algorithm To Find The Largest Value Among N 1 Numbers
10 pages
Unit - 1 Daa
No ratings yet
Unit - 1 Daa
38 pages
Algorithms Lab Manual (SCSVMV DU)
No ratings yet
Algorithms Lab Manual (SCSVMV DU)
66 pages
Binary Search
No ratings yet
Binary Search
44 pages
Algo DataStruct Progr
No ratings yet
Algo DataStruct Progr
55 pages
Introduction to Algorithms
No ratings yet
Introduction to Algorithms
41 pages
Flow Chart For Product of First N Natural Numbers: Syllabus/Lectures/Same/First Grade/programming 1 PDF
No ratings yet
Flow Chart For Product of First N Natural Numbers: Syllabus/Lectures/Same/First Grade/programming 1 PDF
7 pages
DSU Unit 2 Q and A
No ratings yet
DSU Unit 2 Q and A
19 pages
Daa Final
No ratings yet
Daa Final
84 pages
DAA All units
No ratings yet
DAA All units
124 pages
Ilovepdf Merged
No ratings yet
Ilovepdf Merged
16 pages
Daa
No ratings yet
Daa
3 pages
Unit-5 Pps PDF
No ratings yet
Unit-5 Pps PDF
9 pages
Dsa Lab 2
No ratings yet
Dsa Lab 2
9 pages
Foca 1 - Bca
No ratings yet
Foca 1 - Bca
8 pages
DESIGNA AND ANALYSIS OF ALGORITHMS3
No ratings yet
DESIGNA AND ANALYSIS OF ALGORITHMS3
139 pages
PPS Unit-5 Notes
No ratings yet
PPS Unit-5 Notes
20 pages
Bubble Sort
No ratings yet
Bubble Sort
3 pages
Unit 5
No ratings yet
Unit 5
25 pages
Data Stracture Reviewer Prelim
No ratings yet
Data Stracture Reviewer Prelim
10 pages
Unit 2
No ratings yet
Unit 2
5 pages
Sorting Search New
No ratings yet
Sorting Search New
15 pages
Algorithms
No ratings yet
Algorithms
61 pages
Data Structure File
No ratings yet
Data Structure File
310 pages
Unit 1
No ratings yet
Unit 1
28 pages
Advanced Data Structures
No ratings yet
Advanced Data Structures
597 pages
How To Calculate Big O
No ratings yet
How To Calculate Big O
9 pages
Analysis of Algorithm
No ratings yet
Analysis of Algorithm
20 pages
Test1 Key PDF
No ratings yet
Test1 Key PDF
8 pages
DAA - Lab - Manual - 1.1
No ratings yet
DAA - Lab - Manual - 1.1
77 pages
GYENEDRA DSA
No ratings yet
GYENEDRA DSA
42 pages
DSU SEM THEORY
No ratings yet
DSU SEM THEORY
34 pages
Chapter 3
No ratings yet
Chapter 3
30 pages
Assignment ON Algorithm Development
No ratings yet
Assignment ON Algorithm Development
15 pages
DOC-20241106-WA0019.
No ratings yet
DOC-20241106-WA0019.
13 pages
DS Unit III
No ratings yet
DS Unit III
37 pages
chap 1
No ratings yet
chap 1
34 pages
DAA Unit-2
No ratings yet
DAA Unit-2
38 pages
Array Structures
No ratings yet
Array Structures
81 pages
TD 225 PDF
No ratings yet
TD 225 PDF
10 pages
Unit1 Daa
No ratings yet
Unit1 Daa
21 pages
DCA7104 - Analysis and Design of Algorithms - Set1&2
No ratings yet
DCA7104 - Analysis and Design of Algorithms - Set1&2
13 pages
(A) What Is Randomized Quicksort? Analyse The Expected Running Time of Randomized Quicksort, With The Help of A Suitable Example. Answer
No ratings yet
(A) What Is Randomized Quicksort? Analyse The Expected Running Time of Randomized Quicksort, With The Help of A Suitable Example. Answer
14 pages
Exam
No ratings yet
Exam
15 pages
Searching and Sorting
No ratings yet
Searching and Sorting
27 pages
Design & Analysis of Algorithms: Practical File
No ratings yet
Design & Analysis of Algorithms: Practical File
34 pages
Unit - I
No ratings yet
Unit - I
28 pages
Top - Down Design
No ratings yet
Top - Down Design
33 pages
A Short Course in Discrete Mathematics
From Everand
A Short Course in Discrete Mathematics
Edward A. Bender
3/5 (1)
Top Numerical Methods With Matlab For Beginners!
From Everand
Top Numerical Methods With Matlab For Beginners!
Andrei Besedin
No ratings yet
Basic Exercises for Competitive Programming: Python
From Everand
Basic Exercises for Competitive Programming: Python
Jan Pol
No ratings yet
نواتج التعلم اولى ابتدائي كونكت (www.darsenglizy.com موقع درس انجليزي)
No ratings yet
نواتج التعلم اولى ابتدائي كونكت (www.darsenglizy.com موقع درس انجليزي)
2 pages
BBM 212 Principles of Marketing Notes
No ratings yet
BBM 212 Principles of Marketing Notes
24 pages
Rayhan Punia (G802) Brand New - Criterion B Task Sheet GDOC (2023-25)
No ratings yet
Rayhan Punia (G802) Brand New - Criterion B Task Sheet GDOC (2023-25)
9 pages
Final Paper
No ratings yet
Final Paper
4 pages
Case7 Washdale Manufacturing
No ratings yet
Case7 Washdale Manufacturing
4 pages
GenEd 7 STS
No ratings yet
GenEd 7 STS
12 pages
Opportunity To Become A Part of Caa'S Professional Team Situation Vacant Notice No. 08/2019
No ratings yet
Opportunity To Become A Part of Caa'S Professional Team Situation Vacant Notice No. 08/2019
2 pages
Configure or Change Fxos Password Firepower 2100
No ratings yet
Configure or Change Fxos Password Firepower 2100
3 pages
S2 bio 9854
No ratings yet
S2 bio 9854
8 pages
Steel Mesh System For Reinforced Fill
No ratings yet
Steel Mesh System For Reinforced Fill
25 pages
Electric Potential and Potential Energy
No ratings yet
Electric Potential and Potential Energy
7 pages
R - Soccer Problem Statement
100% (1)
R - Soccer Problem Statement
4 pages
Handout 3 - Pronoun-Antecedent Agreement
No ratings yet
Handout 3 - Pronoun-Antecedent Agreement
2 pages
Nooteboom MCO-92VV Low Bed Semi-trailer for Sale Netherlands Raalte, UA42387 3
No ratings yet
Nooteboom MCO-92VV Low Bed Semi-trailer for Sale Netherlands Raalte, UA42387 3
1 page
UNITRONIC ® Li2YCY PiMF
No ratings yet
UNITRONIC ® Li2YCY PiMF
1 page
How To Be Healthy Essay 4
No ratings yet
How To Be Healthy Essay 4
2 pages
C++ STL
No ratings yet
C++ STL
53 pages
ABG Shipyard Scam
No ratings yet
ABG Shipyard Scam
5 pages
Project Report-BSNL Multiplay
No ratings yet
Project Report-BSNL Multiplay
11 pages
MODULE 3 CE 214 Fundamentals of Surveying
No ratings yet
MODULE 3 CE 214 Fundamentals of Surveying
11 pages
Categorical Syllogism
No ratings yet
Categorical Syllogism
29 pages
Atia Group USA Executive Summary
No ratings yet
Atia Group USA Executive Summary
6 pages
Email Id of Controllers PDF
No ratings yet
Email Id of Controllers PDF
10 pages
Matlab
No ratings yet
Matlab
13 pages
NewYears2021FREEBIE 1
No ratings yet
NewYears2021FREEBIE 1
7 pages
Joint Speech-Text Embeddings For Multitask Speech Processing
No ratings yet
Joint Speech-Text Embeddings For Multitask Speech Processing
13 pages
Chartered Quantity Surveyor, Cost Consultant, Independent Adjudicator and Charted Builder
100% (4)
Chartered Quantity Surveyor, Cost Consultant, Independent Adjudicator and Charted Builder
36 pages
Curriculum Vitae: Md. Rayhan Uddin
No ratings yet
Curriculum Vitae: Md. Rayhan Uddin
3 pages
Mortgages 2019
No ratings yet
Mortgages 2019
60 pages
Main Lectures Material ME 223
No ratings yet
Main Lectures Material ME 223
48 pages

Design and Analysis of Algorithm

Uploaded by

Design and Analysis of Algorithm

Uploaded by

Introduction to Algorithms

An algorithm is a finite set of instructions that, if followed, accomplishes a particular task.

In addition, all algorithms must satisfy the following criteria :

Process of translating a problem into an algorithm

Little more detailed solution:

More detailed solution which is a formal algorithm:

Find largest among list of elements

More detailed solution which is a formal algorithm:

Algorithm Largest (a,n)

Little more detailed solution:

More detailed solution which is a formal algorithm:

Little more detailed solution:

More detailed solution which is a formal algorithm:

Little more detailed solution:

More detailed solution which is a formal algorithm:

Little more detailed solution:

More detailed solution which is a formal algorithm:

Recursion has most common applications in Mathematics and computer science.

1.BASIS: This step defines the case where recursion ends.

For Example in Example 3,

Basis is n=1 and

Recursive step is n + Sum(n-1)

Recursive solution for the Sum of nos

First Call for Recursion:

Recursive solution for the Largest of ‘n’ numbers

First Call for Recursion:

Algorithm RecursiveLargest (a,k)

Recursive solution for the SelectionSort

First call of recursion is:

Algorithm RecursiveSelectionSort (a, m)

Recursive solution for the Linear Search

Algorithm linearSearch(a, key, size)

Recursive solution for the BubbleSort

Algorithm RecursiveBubbleSort (a,k)

Recursive solution for the Printing an array in normal order

First call of recursion is:

Recursive Algorithm prin_arr(int a[],int n)

Recursive solution for the Decimal to binary conversion

Recursive solution for the Fibonacci number

Recursive Algorithm fib(int x)

Recursive solution for the Factorial of a given number

Recursive Algorithm fact (int n)

Towers of Hanoi with recursion:

Algorithm TowersOfHanoi (numberOfDisks, x,y,z)

Program step count method – examples:

Algorithm Sum (a,n)

Algorithm Add (a,b,c,m,n)

Step table method:

Statement s/e frequency total steps

Step table for Algorithm RSum

Statement s/e frequency Total steps

Step table for Algorithm Addition of two matrices

Statement s/e frequency Total steps

Fill the blank columns in the above Table

• Posteriori Analysis is aimed at determination of actual statistics about algorithm’s consumption of

c1=1,c2=2 & c3=100

Then c1n2+c2 n is ≤ c3n for n≤98 and

c1n2+c2n is > c3n for n>98

The Common asymptotic functions are given below.

The growth of the functions as below

1< log n < n <n log n < n2 < n3 < 2n <n!

Ex2: The function 3n+2=O(n) as 3n+2≤4n for all n≥2.

Pb1: 3n+3=O(_______) as 3n+3≤______ for all________.

Ex3: 10n2+4n+2=O(n2) as 10n2+4n+2≤11n2 for all n≥5

Pb2:1000n2+100n-6=O(_______) as 1000n2+100n-6≤________for all_______

Ex4:6*2n+n2=O(2n) as 6*2n+n2 ≤ 7*2n for n ≥ 4

Ex5:3n+3=O(n2) as 3n+3≤3n2 for n≥2.

Ex 6:10n2+4n+2=On4 as10n2+4n+2≤10n4 for n≥2.

Ex:2 100n+6=Ω(n) as 100n+6≥100n for n≥1

Pb:3 4n2 – 64 n + 288 = Ω (____)as 4n2 – 64 n + 288≥_______ for all n≥_________ .

Pb:5 n3 logn is = Ω (____)as n3 logn__________for all n≥ ___.

Ex 1 : The function 3n + 2 = Ө(n) as 3n + 2 ≥ 3n for all n ≥ 2 and 3n + 2 ≤ 4n for all n ≥ 2 so c1 = 3,

Theorem: If f(n) = amnm+.......................+a3n3+a2n2+a1n+a0, then f(n)=O (nm)

f(n) = O(nm) (assuming that m is fixed ).

The function 3n+2 = o(n2) since lim 3n+2 = 0.

Ex1: 3n+2 =o(n log n).

Pb1: 3n+3=O(_) as 3n+3≤ for all____.

Pb2:1000n2+100n-6=O(_) as 1000n2+100n-6≤for all_

Ex4:62n+n2=O(2n) as 62n+n2 ≤ 7*2n for n ≥ 4

Pb:3 4n2 – 64 n + 288 = Ω ()as 4n2 – 64 n + 288≥_ for all n≥_______ .

Pb:5 n3 logn is = Ω ()as n3 logn____for all n≥ _.