100% found this document useful (1 vote)
3K views219 pagesAOA
Uploaded by
avm5439Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
100% found this document useful (1 vote)
3K views219 pagesAOA
Uploaded by
avm5439Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
/ 219
Strictly as per the New Revised Syllabus (REV- 2019 ‘C’ Scheme) of Mumbai University
wef. academic year 2020-21 (As per Choice Based Credit and Grading System)
Analysis of
Algorithms
(Code : CSC402)
Semester IV - Computer Engineering
f Includes : >
© Solved Latest University Question Papers Dr. Mahesh M. Goyani
upto Dec. 2019.
) i ~
Be TechKnowledge
Publications Analysis of Algorithms
(Code :CSC402)
Strictly as per the New Revised Syllabus (Rev-2019 ‘C’ Scheme)
of Mumbai University w.e.f. academic year 2020-2021
Prof. (Dr.) Mahesh M. Goyani
Ph.D. B.A., ME. BE.
Assistant Professor,
Department of Computer Engineering,
Government Engineering College. Modasa
Gujarat, India.
MEL35A Price t 250/-
WO
Se TechKnowledge
¥ i
Publications ee
Analysis of Algorithms (csc402)
prof. (Dr.) Mahesh M. Goyani
Semester IV - Computer Engineering (Mumbai University)
© by Author. All rights reserved. No part of this publication may be reproduced, copied, o-
uted or transmitted in any form or by any means, including photocopy
chanical methods, without the prior written permission of ty
Copyright
stored in a retrieval system, distri
recording, or other electronic or met
publisher.
wn that it shall not, by the way of trade or otherwise, be lent, resold
jor written consent in any form of binding cr
ilar condition including this condition b
der copyright reserved above.
This book is sold subject to the conditio
hired out, or otherwise circulated without the publisher's Pr
cover other than which it is published and without a sim!
imposed on the subsequent purchaser and without limiting the rights un
First Printed in India + January 2018
First Edition : January 2021
Maldives, Nepal, Pakistan, Sri Lanka and «
Bangladesh, Bhutan,
ss is unauthoriz
This edition is for sale in India,
rrchase of this book outside of these countrie:
countries in South-East Asia. Sale and pu
the publisher.
ISBN : 978-93-90428-49-6
Pag
‘TechKnowledge Publications
CHT Rete
37/2, Ashtavinayak Industrial Estate, B/S, First floor, Maniratna Complex
‘Taware Colony, Aranyeshwar Corner, |
Narhe, Pune, Maharashtra State, India, Pune - 411 009, Maharashtra state, Indi)
Pune 411041 Ph: 91-20-24221234, 91-20-24225078
Email : info@techknowledgebooks©o™ |
Website : www.techknowledgebooks
i
Near Pari Company
Subject Code : CSC402
Book Code__: ME135A. ‘We dedicate this Publication soulfully and wholeheartedly,
in loving memory of our beloved founder director,
Late Shri. Pradeepji Lalchandji Lunawat,
who will always 6e an inspiration, a positive force and strong support
behind us.
“My work. is my prayer to God”
~ Lt. Shri. Pradeepji L. Lunawat
Soulful Tribute and Gratitude for all Your
Sacrifices, Hardwork, and 40 years of Strong Vision... reface
ar Students,
Tam extremely happy to present the book of “Analysts of Algorithms” for you. | have divided
the subject into small chapters so that the toplcs can be arranged and understood properly. The
1 the chapters have been arranged in a proper sequence to ensure smooth flow of the
topics
subject.
1 present this book in the loving memory of Late. Shri. Pradeepji Lunawat, our source of
inspiration and a strong foundation of “TechKnowledge Publications”. He will always be
remembered in our hearts and motivate us to achieve our new milestone,
J. 8. Katre, Shri, Shital Bhandari, Shri, Arunoday Kumar and
1 am thankful to S
r the encouragement and support that they have extended. | am also
Shri, Chandroday Kumar fi
‘cema Lunawat for technology enhanced reading, E-books support and the staff
thankful to
‘Knowledge Publications for their efforts to make this book as good as it is. We
members of
have jolntly made every possible efforts to eliminate all the errors in this book, However if you find
any, please let us know, because that will help us to improve further.
+ Author
goo —_—_———
Ero
Mumbai University
Second Year of Computer Engineering (2019 Course)
Analysis of Algorithms (cSC402)
Credits
3
Course Code Course Name
csc402 _| Analysis of Algorithms
Prerequisite Courses: Data structure concepts, Discrete structures
Course Objectives :
1. Toprovide mathematical approaches for Analysis of Algorithms.
2. Tounderstand and solve problems using various algorithmic approaches.
3. Toanalyze algorithms using various methods.
Course Outcomes : At the end of the course learner will be able to
Analyze the running time and space complexity of algorithms,
Describe, apply and analyze the complexity of divide and conquer strategy.
Describe, apply and analyze the complexity of greedy strategy.
Describe, apply and analyze the complexity of dynamic Programming strategy.
1
2
3.
4.
5. Explain and apply backtracking, branch and bound.
6
Explain and apply string matching techniques,
a Detailed Contents Howls
rh Introduction [y
matical hagrPlexity Growth of function, Big-0h.
ao background for algorithm analysis.
NP. NP-Hard, NP-Complete Analysis of
12
19
(Refer Chapters 1 and 2) Hours:
nding minimum and maximum
of Binary search.
(Refer Chapter 3)
2d
Reneral method, Merge sort, Quick s
Analysis, Analy:
algorithms and thei
3 Greedy Method Approach
3.1 | General Method, Single source shortest path: Dijkstra Algorithm, Fractional
Knapsack problem, Job sequencing with deadlines, Minimum cost spanning
tr (Refer Chapter 4)
- Dynamic Programming Approach
4.1 | General Method, Multistage graphs, Single source shortest path: Bellman
Kruskal and Prim’s algorithms,
Ford Algorithm,
All pair shortest path: Fl
Problem0/1 knapsack
_| common subsequence.
5 Backtracking and Branch and bound
5.1 | General Method, Backtracking: N-queen problem, Sum of subsets, Graph
yy Warshall Algorithm, Assembly-line scheduling
coblem, Travelling Salesperson problem, Longest
(Refer Chapter 5)
coloring.
\d Bound: Travelling Salesperson Problem, 15 Puzzle problem.
(Refer Chapter 6)
5.2
6 String Matching Algorithms 4
atching algorithm, The Rabin Karp algorithm, The Knuth-
(Refer Chapter 7)
6.1 | The Naive string:
Morris-Pratt algorithm .
Analysis of Algorithms Lab (CSL401)
Lab Code Lab Name Credits
‘CSL401 | Analysis of Algorithms Lab i:
Prerequisite : Basic knowledge of programming and data structure
Lab Objectives :
4. Tointroduce the methods of designing and analyzing algorithms
2. Design and implement efficient algorithms for a specified application
3, strengthen the ability to identify and apply the suitable algorithm for the given real-world problem.
4, analyze worst-case running time of algorithms and understand fundamental algorithmic problems.
Lab Outcomes : At the end of the course, the students will be able to
1, Implement the algorithms using different approaches.
2. Analyze the complexities of various algorithms.
3, Compare the complexity of the algorithms for specific problem. Rien
Implementation can be in any language. yl
Suggested Practical List : 7
ist of Laboratory EXP!
Sr.No.
Suggested Experiment List
1
Introduction
14
Selection sort, Insertion sort
Divide and Conquer Approach
24
Finding Minimum and Maximum, Merge sort, Quick sort, Binary search
Greedy Method Approach
31
Single source shortest path- Dijkstra
Fractional Knapsack problem
Job sequencing with deadlines
Minimum cost spanning trees-Kruskal and Prim’s algorithm
Dynamic Programming Approach
4d
Single source shortest path- Bellman Ford
All pair shortest path- Floyd Warshall
0/1 knapsack
Travelling salesperson problem
Longest common subsequence
Backtracking and Branch and bound
51
N-queen problem
Sum of subsets
Graph coloring
64
String Matching Algorithms
The Naive string-matching Algorithms
The R
Karp algorithm,
‘The Knuth-Morris-Prate algorithm WF _Analysis of Algorithms (MU) 1
Table of Contents
Risers
Chapter 1: Introduction to Analysis of Algorithm
1-4t01-27
1d IntFOdUCIOR enn A
1.4.1 Whatis Algorithm?, io
1.1.2 Properties of Algorithm. Ad
143 Howto Write an Algorithm 42
12 Performance Analysis ncn aia)
12.1 Space Complexity . 13
122 Time Complexity. 15
123. Growth of Function... 1.6
1.2.4 Asymptotic NOtAtOM nen 17
1.2.4(A) Big Oh. 18
1.2.4(B) Big Ome ern 19
12.4(C) Big Theta.. 1.9
13 Mathematical Background for
Algorithm Analysis..
13.1 Framework for Analysis of
Non-Recursive Algorithms.
1.32 Framework for Analysis of
Recursive Algorithm:
14 Complexity Class.
14.1 Introduction to Complexity Theory
142 Problems...
143 NPProblem
144 Difference between P and NP.
145 NP-Completeness and Reducibility..
146 NP-Completeness Proofs.
14,6(A) Vertex Cover Problem..
14,6(B) Clique Problem
1.47 NP-Hard Problem..
148 Comparison between NP
Hard and NP Complete
15. ‘Analysis of Selection Sort and Insertion Sort... 1-22
15.1 Selection Sort... 1.22
152 Insertion Sort. eed
Chapter 2: Recurrence 21 to27
2a ‘The Substitution Method a2
22 Recursion Tree.. 24
23 Master Method men BS
Chapter 3: Divide and Conquer 344 to 3-29
3a General Method ... 31
3.1 Introduction... : 34
31.2 Control Abstraction... a 32
313° EMMiciency Analysis. 32
a 33
32 Merge Sort 3:3
33 Quick Sort... 3.9
34 Binary Search.. 3:23
35 Min-Max Problem : 3-26
36 Strassen's Matrix Multiplication 3-28
ee ————
Cheplor4_Greedy Algorithme ere a88
<1 General tho #1
$12 cond Abtcton 4
413° Guratersi +2
tia Apleatons of Greedy Approash—_—
425 Dvdeand Conuer vs. credy Algor.
416 Dyan Popammng Vs red approach
42, Single Source shortest Pl
43° Krapsack ble.
431 FocionlKapsack Problem
44 lobsequencg
453s agontim
454 —_biflerence beeen Prins and
Seeoyae satin
462 Stonge one Tapes
Chapter; Dynamic Programming 1108-1
Sr Gener Netod
2
52 Pancpleofopinaliy
53 Bloentsof Dynamic hegrmming Tabi,
Backtracking Vs Branch and Boung_ SS
W_ Analysis of Algorithms (MU)
55 Multistage Graph annem
56 ‘ingle Source Shortest Path em
87 Al Pair Shortest Path.
58 Assembly Line Scheduling
59 0/1 Knapsack..
594 First Approach.
5.10 Travelling Salesman Problem 5-45
ett Langest Common SUBSeqUERCEmmnnwnm—een 5
Chapter 6: Backtracking & Branch and Bound
6-1 t0 6-35,
64
614
61.4(4)
61.108)
61.2 Control Abstraction
61.2(A) Recursive Backtracking Method
6:1.2(8) Iterative Backtracking Method
613 Applications of Backtracking
614 8-Queen Problem.
615 SumofSubset Problem.
616 Graph Coloring
62
621
‘Applications of Backtracking
Control Abstraction...
624(A) LCSearch. __™
624(8) Control Abstraction for Least Cost sean,
62.4(C) Bounding. on Seah
62.4(0) Control Abstraction for
FIFO Branch and Bound...
625 15 Puzzle Problem. =
626 Travelling Salesman Problem.
62.6(A) _LCBB using Static State Space Tree... =
626(B) | LCBB using Dynamic State Space Tree,
627 Comparison between Divide and Conquer,
Dynamic Programming and Backtracking_
epi
ee
Chapter 7 :_ String Matching
74 Introduction.
72 ‘The Naive String Matching Algorithms —_
73 ‘The Rabin Karp Algorithm.
74 String Matching with Finite Automata..
78 The Knuth-Morris-Pratt Algorithm.
+ Lab Manual set Introduction to
Analysis of Algorithm
Analysis of selection sor, Insertion sort
1.1 Introduction
= The term Algorithm was coined by the Persian
mathematician al-Khowarizmi in the ninth century, The
algorithm is a set of rules which are used to solve real-
life problems. The algorithm provides a loose form of a
solution in a pseudo-programming language.
~ Given the algorithm, itis easy to program the solution. It
bridges the gap between natural language description of
the solution and syntax of programming language.
= The first ever algorithm was developed by Babylonians
for factorization of a number and to find roots of the
equation. Euclid had proposed a famous algorithm for
finding greatest common divisor (GCD) of two numbers.
= We can treat an algorithm as a set of finite instruct
which solves a particular problem when applied it Is
applied to that problem with legal inputs.
4.1.1. What is Algorithm?
[a Define algorithm. (2 Marks)
Definition
The algorithm is set of rules defined in specific
‘order to do certain computation and carry out
some predefined task. It is a step by step procedure
to solve the problem.
= Initially, the solution to the problem is thought as a
natural language description, whose syntax Is too far
from the programming language.
Performance analysis, Space, and time complexity, Growth of
Mathematical background for algorithm analysis. Complexity class
function, Big-Oh, Omega, Theta notation,
Definition of P, NP, NP-Hard, NP-Complete,
= Before the actual problem solved in a programming
atural language description of the solution is
first represented as an algorithm. The algorithm is then
language, n
converted to code.
= The process of solving a problem is depicted in the
Fig. 1.1.1.
Input
4
Problem vee Program
Naural Paws Poganl
sanguase code ‘anowsoe
out
os
conéét ort
‘wait any
Fig, 4.1. : Design of solution to the problem
= If the algorithm is correct, then the program should
produce correct output on valid input, otherwise, it
should generate an appropriate error message. For
example, to find the division 4/B, correctly written
program would return value of A/B for B > 0, and it
would show the error message like “Invalid divisor” for
4.1.2 Properties of Algorithm
@. Discuss various characteristics of the algorithm.
(4 Marks)
(4 Marks)
Q. Explain properties of the good algorithm.
= A good algorithm should have following properties /
characteristics : 2210 Anay
ot Ago (MU ky
: {11 How to Welte an Algoritin,
Fig. 1.1.2 : Characteristics of good algorithm
4) Input
Algorithon may take zero or more input arguments
Depending on the problem, the input may be a scalar,
tree, graph or some other data structures,
vector, ar
2) Ontput
Algorithm reads input, processes it and produces. at
east one output, Depending on the problem being solved,
the output may ofthe form scalar, vector, array, tree, graph
‘or some other data structures.
3) Definiteness
AL instructions in algorithm should be unambiguous
and simple to interpret. There should not be multiple ways
to interpret the same Instruction, Instructions should be
precise and concise,
4) Finiteness
Every algorithm must terminate aftera finite number of
steps. If algorithm contains a loop, the upper bound of the
loop must be finite. Recursive calls should have
defined base case to terminate the algorithm,
a well.
5) Effectiveness
The algorithm should be written with a haste set of
instructions. The operations should be
perform exactly using basic set, ust lke
them with pen
Performed using a combination of baste Instruction, Fer
example, multiplication should he performe,
and addition, sorting should be
comparison, swapping e
basic enough to
fone can perform
lons should be
id paper. Comples oper
“t using loop
carved out using looping,
eT ——
, Discuss the rulos to write an aigoritig
Q. What are the general rules followog whi 6
algorithm? ay
&
te agorthn basiealy Consists of yy
body. H
~ Heal pat consists of algorithm name doy
problem being solved, input to the a
espected output I may als lacie h ga
input arquent amd output variable he ge
provides clear idea to the user about te aya,
thealgorithm,
= Body part Includes a logieal sequence of gy
involving va
expressions, loops, breaks, function calls et
Algorithm namo, Description,
inpuit and output
Logical sequence of
statements to solve problem
lous constructs lke conltonal yy,
Fig. 1.1.3 : Structure of algorithm
An algorithm is a lucid form of program and ita
have rigid restrictions of syntax. One can writeanalgs
Using his own terminology.
However, some of the common rules follow!
writing algorithms, w
1, The algorithm should start with the
Algorithm, followed by name of algoritha, foe
alist of arguments to be passed to the algorithi
Algorithm FIND_SUM(A, B)
2 Comments start with // sign, In very next I
o
should specify the description and input-ott*"
algorithm,
ich are stated below:
es
// Problem Description : Add two integer
nis
7/Uput : Two numbers A and B on which st
performed
——__
77 Output : Sum ofgiven two numbers
ee WF Analysis of Algorithms (MU) 13
Introduction to Analysis of Algorithm
Examples of how to write algorithms
3, Next, comes body part, which contains various logical
statements in proper sequence. These statements may | Ey. 1.4.1: Write an algorithm for finding the factorial of
contain control statements, loops, expressions ete. number n.
4. Compound statements are enclosed within the opening | Soin. :
and closing curly braceie.{...). “Algorithm FACTORIAL (a)
5, Use left arrow forassignment : C— A+B, 1] Deseription : Find factorial of given
6 Array index usually starts with index 0 oF 1 M input: Number n whose factorial is to he computed
11 Output : Factorial of n= nx (n= 1) X02 2X1
7. Relational operations are performed using relational
operators like <, zands
8. Logical operations are performed using logical
operators like and (a), or (v) and < not (—). else
9. Inpat and output are performed using read and write |] "=" “FACTORIAL(a= 1)
statement.
read (A) / read “A”
write (A) / write “A” or print (A) / print “A”
10, Control statements are written as follows
if (condition) then Statement end
if (condition) then Statement else Statement end
Multiple statements are enclosed within {...}
11, While loop is written as follows :
while (condition) do
1
Do some work
}
‘Sometimes curly braces are omitted and block is closed
with end keyword.
Ex. 1.4.2 : Write an algorithm to perform matrix
mutipication
Soin.:
“Algorithm MATRIX_MULTIPLICATION(A, B)
1] Description : Perform matrix multiplication of two
mares.
{Input : Two matrices A and B of wi
1 Output : Resultant matrix containing multiplication of
Aand B
fori Itondo
for} 1 ton do
chi] eo
fork @ 1 1ondo
CEL — CLG] + ALTE * BOG)
end
xn,
end
end.
while (condition) do
Do some work
end
12, For loop is written as follows
for index « FirstIndex to LastIndex do
{
Do some work
}
Som:
with end keyword,
es curly braces are omitted and block is closed
for index € FirstIndex to LastIndex do
Do some work
end
| 12 Space Complexity
= However, there is no strict rule to follow these
standards. Algorithm syntax may vary from person to
person.
1.2. Performance Analysis
@. What are the basic components which contribute to
space complexity? (4 Marks)
@. What do you mean by space complexity of an
algorithm? How do we measure the space comploxity
of an algorithm? Explain with suitable example.
(4 Marks)
Q. Explain space complexity in detail
Teeatawutetgt Y& Analysis of Algorithms (MU)
fen
= space complexity is very important nation of efficiency
14 Introductiy
21
Example 1: Addition of two seal ye
a
Algorithm ADD_SCALAR(A, py
seal I Description: Pesform aitintc aii
Definition [input : Two scalar variables 4 ang y '
Problem-solving using computer requires memory 1 Output : variable C, which holds the ada
to hold temporary data or final result while the vton y
program is in execution (The amount of memory |)) C-A+B
required by the algorithm to solve given problem i |) return C
called space complexity of the algorithm.)
Controlling components|
of Space Complexity
1. Fixed size components
2. Variable size components
Fig. 1.21 : Controlling components of
Space complexity of algorithm
‘The space complexity of the algorithm is controlled by
two components:
1) Fixed size components
It includes the programming part whose memory
Fequirement does not alter on program execution. For
example,
(©. Instructions
© Variables
‘© Constants
© Arrays
2) Variable size components
Ie Includes the part of the program which whose size
depends on the problem being solved. For example,
© Size of loop
© Stack required to handle recursive call
© Dynamic data structures like linked list
We use the notation S(n) to specify the space co
‘of the problem for input sizen, The term n ig
the size of input or the problem size,
The notion of space complexity is expla
following examples:
—__| sence
mplexity
treated as
ined with
The addition of two scalar numbers en,
memory location to hold the resul 4,
complexity ofthis algorithm is constant hese,
Example 2 : Addition of two arrays
‘Algorithm ADD_ARRAY(A, B)
[Description : Performa element
two arrays
IM Input : Two number arrays A and B
1/ Output : Array C holding the element-vise une
AandB
fori 1ltondo
Cli] Abi] + BE
end
return C
se arithnetns
= Adding corresponding elements of two 2138
size n requires extra n memory locatiots
result. As input size n increases, required =
the result also grows in the linear order
the space complexity of above code s#
S(n) = O(n).
Example 3 : Sum of elements of array
Algorithm SUM_ARRAY_ELEMENTS(\)
!/ Description : Add all elements of aay A
Input : Array A of size n
// Output : Variable Sum whieh holds the i?
clements,
Sum <0
fori —1tondo
Sum Sum + Afi
lend °
return Sum
5
4
The addition of all array eleme®
rocatot
extra variable (j
sum, th
one memory
independent of array s##° WF Analysis of Algorithms (MU) 15
Introduction to Analyato of Algorithin
‘Thus the space complexity of the algorithm is constant
and S(n) = 0(1).
jee Time Complexity
‘@. How do we analyze and measure the time complexity
of an algorithm? (4 Marks)
@. What do you mean by time complexity of an
algorithm? How do we measure the time complexity of
an algorithm? Explain with suitable example.
(4 Marks)
@. Explain time complexity in det
= Goodness of algorithm is often determined by the time
complexity. Time complexity is the most fundamental
component of analysis framework.
Definition
The valid algorithm takes a finite amount of time
for execution( The time required by the algorithm
to solue given problem is called time complexity
of the algorithm. Time complexity is very useful
measure in algorithm analysis.)
‘Time complexity is not measured in physical clock ticks,
rather it is a function of a number of primitive
operations. Primitive operation is the most frequent
operation in algorithm
—(We use the notation T(n) to symbolize the time
‘complexity of the problem for input size.
= The notion of time complexity is explained with
following examples:
Example 1 : Addition of two scalar variables
“Algorithm ADD_SCALAR(A, B)
‘J Description : Perform arithmetic addition of two numbers
I Input : Two scalar variables A and B
J] Output : variable C, which holds the additi
n of Aand B
CeA+B
return
‘The sum of two scalar numbers requires one addition
operation. Thus the time complexity of this algorithm is
constant, so T(n) = O(1).
Example 2 : Perform addition of two arrays
Algorithm ADD_ARRAY(A, 1)
11 Deseription : Performa elem
of two arrays
Input
1 Output : Array G holding, the «
and B
fori 1 tondo (fone
comparison
Chi] CAL + BM
wo muumbor arrays A andl 1 of length a
i an of array A
ininligation, 1 ineeementy
1) n addition und 1 asiganvent
return C
hove code, addition array
able 1 Is
nd
‘As it can be observed from
elements required iterating loop n times.
initialized once, the relation between control variable 1
ate checked n times, and I is incremented n times. With
the loop, addition operations
performed n time,
and assignment
‘Thus the total time of algorithm is measured
‘T(n) = 1(inltilization) + n{copmparis
+ Increment addition + assignment)
= 1t4n
While doing efficiency analysis of the algorithm, we are
interested in the order of complexity in term of input size n.
What is the relationship between Input size n and the
‘number of steps to be performed? So all multiplicative and
divisive constants should be dropped. Thus, for given
algorithm T(n) = O(n)
Example 3 : Sum of elements of array
‘Algorithm SUM_ARRAY_ELEMENTS(A)
11 Description : Add all elements of array A
UM Input : Array A of size
um which holds the addition of array
Sum <0
for i<-1tndo
Sum € Sum + Ali]
end
= The addition of all array elements requires n additions
{we shall omit the number of comparisons, assignment,
initialization etc, to avold the multiplicative or additive
constants).
Tecate Introduction 1 Ana,
16 oy,
WF Analysis of Algorithms (MU) 7 a
— Anumber of additions depend on the size of en Emnelercy ee, Hi
grows in the linear order of input size. Thus the ae Br ple
complexity of above code is T(n) = O(n). ;
~ Time and space complexity of discussed problem Is
Binary search ~~
Logarithmic | logn | insert / delete
ler
compared in the Table 1.2.1 binary search tee.
Table 1.2.1 : Comparison of space and
time complexity for various problems Linear search
Insert nod
PER cnn Time nr node at the 4y
s(n) Tn) Linear a linked list
Find minimum / ma,
Add scalar ou) ow) element from array
Add two om on) Merge sort
ease) Binary search
Tal ol nlogn
Add array on) (n) nlogn 8 | uicksort
elements
Heap Sort
1.2.3 Growth of Function
Q. Define order of growth. List various efficiency classes
Selection Sort
with example, Show the. relationship between Bubble sort
efficiency classes, (Marks) | || ou sdratc he
Definition Find maximum element
The efficiency of the algorithm is expressed in term 2D matrix
of input size n. The relationship between input size
it Cubic n Matrix Multiplication
and performance of the algorithm is called order u
of growth. Finding power set of
BP
~ Order of growth indicates how quickly the time required Find optimal solution
by algorithm grows with respect to input size, Exponential 2" | Knapsack problem
~ For input size n the algorithm may execute a number of Solve TSP using 47"
steps in order of log n,n, n,n® or something else, programming
~ Efficiency classes are categorized into differe,
as shown in Table 1.2.2,
nt classes, Generating permutations
given set :
Solve TSP problem ©
EMciency | Onter of HL] J brute force approach_—
Table 1.2.2 : Various efficiency classes Factorial nl
class | Srowth Example EtMciency classes are sorted as: ;
ae oft . , the term 4n ~ 32, becomes insignificant.
So the growth of fn) is limited by the n® term,
are made while doing
Following assumptions
complexity analysis.
Assumptions
‘The actual cost of operation isnot considered.
Abstract cost cis ignored : O(c. n’) reduces to O(n*)
Only leading term of polynomial is considered :
O(n? + n) reduces to O(n?)
Drop multiplicative or divisive constant if any : O(2n?)
and O(n?/2) both reduces to O(n’).
Various notations like Big Oh (0), Big Omega (0), Big
‘Theta (@), Little Oh (0), Little Omega ((d) are used to
describe the asymptotic running time of the algorithm.
cannot be more than its asym
random sequence of data, forany
Definition
Let fs) and £01) 470 1100 notre,
indicating running time of two ep hy
say, a(n) is upper bound of fin) if yy, Mil
positive constants ¢ and Mo such thy
ign) for all n 2no, It is denoted
In Fig. 1.24, Horizontal axis represen
ee ay
Osh,
5 fn) <4
Ay
Dr Obj,
the vertical axls represents growth a
ont,
required to solve the problem of size n) oy,
eon)
Growth of
function
Po
Fig. 1.2.4 : Upper bound
Definition
For small input size, there may be many os
between the growth rate of fin) and exist
1n becomes significantly large, fin) grow
slower than eg(n). This value of n is’
crossover point and is denoted as nx
Loose bounds
All the set of functions with growth rate hi"
actual bound are called loose upper bound oth?
23 = O(n) = O(n?) = O(n") = O("))
6n+3 (n8) = O(n!)
3n%*2n+4 =
(nt)
2n'e anes = O(2%) =0(n1)
Incorrect bounds
Al the set of functions with a growth F# int
actual bound are called incor ra
6n+3 of)
2
aa 2n+4 # O(n) e0(1)
m+ 4045 # O(n?) O(n) + 0(1)
For fuention f(n) =2n? +43 oe
€) = O(n) = o(m)= 029209
sct bound o!
SS -,.,_
£00) * O(mtog n) + O(n) + o(lo8")”
_——¢r W Anaysis of Algorithms (MU) 12 Intoduction to Analysis of Algorithm
1.2.4(B) 1.2.4(C) Big Theta
Q. Define and explain @ notations. Give examples. {Q, Define and explain © notations. Give examples.
(2 Marks) (2 Marks)
This notation is denoted by ‘f, and it is pronounced as,
“Big Omega’. Big Omega notation defines lower bound
for the algorithm, It means the running time of
algorithm cannot be less than its asymptotic lower
bound for any random sequence of data.
io)
Grovtnat
com
"%
Fig. 1.2.5 : Lower bound
Definition
Let fn) and g(n) are two nonnegative functions
indicating running time of two algorithms. We say
the function g(n) is lower bound of function fin) if
there exist some positive constants c and no such
that 0 ny
Osc+g(n) s(n)
Osc+g(n)s10n?+2n+1
and ny
0s 10n?s 10n? + 2n + 1,9 true, forall n2 1
{{n) = 2 (g(n)) = 2 (n?) for c= 3,ng=1
f(n) = © (g(n)) = © (n*) for cs = 10,c2= 13,4
3. Mathematical Background for
Algorithm Analysis
Before we start analyzing algorithm, we wilt
some important mathematical formulas, which
to simplify the computation further.
lathematics to simplify the summation
= 1+1+14..¢1=n=0(0)
n 5 a
Di s14243+ tne a
n
(n?)
14 2k4 38440! WB anaysis of Algorithms (MU) 11
1.3.1. Framework for Analysis of
Non-Recursive Algorithms
@. Discuss the general plan for analyzing efficiency of
ron-rocursive algorithm. (6 Marks)
@. Explain the framework of efficiency analysis of
non-recursive algorithms with sultable examples.
(6 Marks)
= Finding complexity of the non-recursive algorithm
simpler than that of recursive algorithms. A number of
primitive operations define the complexity of the
algorithm. By following below steps we can find the
complexity of non-recursive algorithms
©. Determine size of problem / input
© Find out primitive / elementary operation
© Find count of primitive operations for best,
worst or average case.
© Simplify the summation by dropping
multiplicative and divisive constants of highest
degree polynomial term in sum,
Ex. 1.3.4 : Determine the complexity to find the sum of
elements of the array.
Soln. :
Soin,
Algorithm BUBBLE, SORT(A, n)
1 Deseription : Sort the given numerieal data
UHnput Atay A of randomly place a ele
1/ Ouiput : Sorted sexquence of input data
for i 1 1ondo
for j 1 wn~i~1do
if Alj] > Alj + 1 then
‘Move largost element at
tho ond of unsorted iat
wap (ALi), Ali)
Algorithm SUM_ARRAY(A, n)
1) Description : Find the sum of elements of array
Mnpat : Array A of length n
1/ Output : Variable Sum which holds the summation of array
elements
‘over length of array A
Sum <0
fori ltondo
Sum < Sum + Afi]
‘Add each element in patil sum
end
print “Sum of Array Elements: ", Sum
Step 1: Size of problem is n because length of array is"
Step2: Primitive operation is addition: Sum = Sum +
Ati
Step3: For loop iterates, n time and hence addition is
performed n times, so T(n) = O(n)
Tin) = Dia14141+..mtimes = O(n)
1
Ex. 1.3.2 : Find complexity of bubble sort
Step1: Size of proble
Step 2: Primitive operation is comparison
loop, inner loop
Step3: For each instance of outer
iterate (n - 1) times.
t loop does n ~ 1 comparisons, for
For = in
Inner oop doesn 2 comparisons and soon
The) + (1) + (0-2) ¢ at 3424
nt
FE tes 243+)
Len-1y2 Bt = * = on’)
‘Ex. 1.3.3 Wile an algorithm for searching an olomont in
array of size n, Calculate complexity of this algorithm.
Soin. :
= We will discuss and derive the complexity of linear
search technique to search an element from an array of
sizen.
= Linear search Is a very simple way of searching an
element from the list. Let Key is the element that we
want to search, The key element is compared with on
by one all index locations of A. Algorithm halts in two
ed,
cases : Key element is found or entire array Is ses
nis shown below :
= Algorithm for linear
‘Algorithm LINEAR_SEARCIH(A, Key)
1] Deseription ; Pesform linear search on array A to search
clement Key
I Iopat : Array of length 1
1/ Output : Success / failure mess
flag — 0
fori C1 tondo Introduetio,
ty
o
ICAfi] > Max, algorithen updates y,
Ue ify
Ai then a to All. The process Is repeated oye, is ‘
print “Element Found on Location” 1 pseudo code of the process is ven hey, Mi
flag 1 Set flag if Key fs found ‘Algorithm FIND_MAX(A) Tim,
pees 11 Description: Find the maxim ly
[Input : Array A of length “i
1 Ourput: Variable Mas olny the macy ‘
array A
Max @ A[]]
fori 2tondo
if Max < Ali] then
Complexity analysis Mace All Unisonan
Best case bad Smaller an
‘The algorithm needs a minimum number of || end
int “Maximum Element of Array A is", Max
‘comparisons if the key element is on the first position. In
the best cas, the size of input array does not matter. Inthe | oor eyity analysis
best case, the algorithm does only one comparison
irrespective of array length. Hence the running time of the
linear search in the best case is, T(n) = (1). problem size is reduced by 1. Recurrence frthiy
formulated as,
T(n) = T(n-1)+1
In every iteration, algorithm does one con:
Worst case
‘The algorithm does a maximum number of comparisons
if the key element is on the last position of the array or itis | Let us solve this using iterat
not present at all, The entire array needs to be scanned, T(n-1) = T(n-2)+1
Numbers of comparisons linearly grow with the size of the
Input. Hence the running time oftinear search in worst case | > TC) = [M(m=2) + 1} 1=T(n-2)+2
T(n-3)+1
ts, T(n) = O(n) T(n-2)
=> Tn) = [T(n-3) +1) +2=T(n-3)+3
After k Iterations,
n method,
Average case
‘The average case occurs when an element is neither on
the first location norat lat. The key element may be near to T(n) = T(n-k) +k
the beginning of array or it may be towards the end, or it For k =n
may be somewhere near the middle, So on an average, the T(n) = T(n-n)+n=T(0)+0
algorithm does (n / 2) comparisons, co fii!
. st for solving problem of size 0 is deft
Thus,T(0) = o($) = 04m) =0
The time complexity of e Hence, T(n)
me compleaty of al three cases is depicted in the | (a) = O(n) “ab
following table. x. 1.3.5 : Write an algorihm to delat an
linked lst. a runt
| Also, mention the worst cas?
Bestcase | Average case | Worst cage operation.
Soln.
st
To delete the element from the lskel
et
‘averse the list to search the element 0 ; if
ie
Ex, 1.3.4 : Write an algorithm to find
unsorted array of size n. Caleulat
algorithm. ; :
Soin. : Rill be traversed unui a node withthe 7
‘und oF end of the list is reached.
To find the maximum element, rst Max tg
‘hen Max's compared with each element ofr? AU
y. y ¥F Analysis of Algorithms (MU)
1.13
Introduction to Analysis of Algorithm
Algorithm for the given operation is described below:
‘Algorithm DELETE_NODE(HEAD, Key)
1 Description: Delete node: from the linked list having value
Key
{/ Input: Linked list surting at HEAD and Key
1/ Output: Linkeil lst after deletion of node with value Key
Temp = HEAD // TEMP points tothe first node in lst
#/ erate until reach to second last node or the node before
the node to be deleted
while Temp —> Next Next # NULL && Temp —> Next >
Data # Key do
Go up to second last node
Temp =
end
1) node tobe deleted is ot the last node
Key && Temp — Next
remp —> Next
if Temp — Next > Dat
Next # NULL then
Hold = Temp — Next
‘Temp —> Next = Hold — Next
Free(Hold)
‘Stop on node before
the node to be deleted
1) inode to be deleted is last node
else if Temp — Next > Dat
Next == NULL then
Hold = Temp — Next
‘Temp — Next = NULL
(Hold)
Key && Temp — Next >
1 key docs not exist
HAL node with yi
else
print “Node not found”
end
‘The problem is similar to linear search. On each
iteration, problem size is reduced by 1, and one comparison
is done. Thus, the recurrence for the above algorithm can be
formulated as,
‘T(n) = T(n-1)+1
Let us solve this using iteration method,
Tn 1) = Tn-2)+1
=> Ta) = [Tn-2)+ 1] 41=T(-2)4+2
Tn-2) = T(n-3)+1
=> Ta) = [T(n-3) +1]+
=~ 3) 43
After kiterations,
T(n) = Tin-K)+k
For k=n
Tn) = T(n-n)+n=T(0)+n
Cost for solving problem of size 0 is definitely 0, so T(0)
=o
Hence, T(n) = O(n)
Worst case for the problem occurs when the node to be
deleted is at the end of the list or itis not present at all. For
the list of n nodes, the worst case running time would be
T(n) = O(n).
Ex. 1.3.6 : Consider the following algorithm.
ALGORITHM sum (n)
11 Input: A non-negative integer n
seo
fori=1tondo
SeS+i
return §,
i) What does this algorithm compute ?
li) What is its basic operation ?
ji) How many times the basic operations executed ?
iv) What is the efficiency class of this algorithm 7
y) Suggest an improved algorithm and incicate its efficiency
class. If you cannot do it, try to prove that it cannot be
done,
Soin.
1) This algorithm computes the summation of numbers
from 1ton
li) The basic operation is an addition, ie, computing sum
fil) Basic operation executes m times
iv) Ast performs n basic operations, efficiency class of this
algorithm is O(n).
v) An improved version ofthe algorithm
‘Algorithm IMPROVED_SUM(n)
[] Description: Add first mumbers
J AAnput: Number
1/ Output: Variable Sum holding the summation of first
su numbers
Suni Gn * (n+ 1)/2
return Sum
‘The complexity of this algorithm is O(1), it performs
only one computation to find the sum of first m natural
numbers.
© “Analysis of Algorithms (MU)
Forn=6,
Using algorithm SUMQ):1+2+3+4+5+6=21
Using Algorithm IMPROVED_SUMQ
nt(n+1)/2=(6"7)/2=21
1.3.2 Framework for Analysis of Recursive
Algorithms
@. Explain the framowork of efficiency analysis of
recursive algorithms with suitable exampk
(6 Marks)
— By following steps, we can find the complexity of
recursive algorithms:
= Determine size of problem
Identity primitive operation
— Find count of primitive operation in each call
= Set up and solve recurrence equation using appropriate
method.
Ex. 1.8.7 : Write @ recursive algorithm for Tower of Hanol
problem, setup its recurrence and soWve it
Soin. :
— The tower of Hanoi is very well known the recursive
problem. The problem is based on 3 pegs (source,
auxiliary and destination) and 1 disks. Tower of Hanol is
the problem of shifting all n disks from source peg to
destination peg using auxiliary peg with following
constraints
= Only one disk can be moved ata time.
‘A Larger disk cannot be placed on smaller disk
‘The initial and final configuration of the disks is shown
in Fig. P.1.3.7(a) and Fig, P.1.3.7(b), respectively
—s
Soren po
1008 estat pg
Fig. P. 1.9.7(a) : Initial state of Tower of Hanot
A
octnaten pag
Baron Poo Taos a
Fig. P. 1.3.7(b) : Final state of Tower of Hanoi
4
Introduction 10 Analysis
Jen number of disks on soure,
©
= There can bs
trace the problem for n= 3 disks. The
solution is :
step 1: Move disk
a
‘oun 98
C from the sre peg to dst peg
‘salary beg Desire,
step 2: Move disk B from the se peg to aux peg
a el
aaa ‘uaiary po Denar
step 3: Move disk C from the dst peg to aux peg
op
‘Sour Pe emaary pe
Desa
‘Step 4: Move disk A from the src peg to dst peg
=
“ausieyDo9
Step 5: Move disk C from the aux peg to sree
Desa
Sours pee
—E]__ a1 ad
Seu
Peo ‘sary peg
Step 6 : Move disk B from the aux peg to dst pes
cS
oon
—f__
Sours peg
‘Aamiary peg
Step 7:
'eP 7 : Move disk C from the sre peg to dst pes
oe ‘Analysis of Algorithms (MU)
recursive approach is the best suitable for solving this
problem, The recursive formulation for tower of Hanol is
givenas,
HANOI{source, aux, dest, n) =
(en from sre to dst inet
HANOI (sre, dst, aus, n = 1)
Jaxon i. dst. 1)
HANOI(aus, ste, dst,n= 1) _ otherwise
Algorithm HANOM (sr, ay
1 Description: Move mdi
pee
{np
1/ Output: n disks on destination peg
‘peg to destination
3 pages, anal clisks on souree peg
HANOMsre, dest, aus, n= 1)
HANOI (re, aus, dst 1)
HANOI(aux, ste, dest, n= 1)
end
Step 1: Size ofproblem isn
Step2: Primitive operation is to move disk from one
peg to another peg
Step 3: Every call makes other two recursive cals with
problem size n- 1. And each call corresponds to
‘operation, so recurrence for this
‘one primiti
problem can be set up as follow :
T(n) = 2T{n-1) +4 a
Let us solve this recurrence using forward and
backward substitution:
Substitute n by n = 1 in Equation (1),
Ten=1) = 2T(n-2)+1,
By putting this value back in Equatton (1),
Tn) = 2[2T(n-2)+ 1]+1
= 2T(n-2)+241
= 2T(n-2)+ (2-1)
Similarly, replace n by n ~2 in Equation (1),
‘T(n=2) = 27(n-3)+ 1,
(2)
From Equation (2),
Tn) = 2f2T(n- 3) +A} He
= 2T[n-3)4 224241
Introduction to Analysis of Algorithm
= 2°T(n~ 3) +(2!-1)
In general,
‘T(n)
By puttingk=n- 1,
T(n) = 24(TCA)] + (22-1)
2aT(n-k) + (241)
T(1) indicates problem of size 1. To shift 1 disk from
source to destination peg takes only one move, so
Ta)eL,
Tn) = 214 (2-1)
na
02")
Ex. 1.3.8 : Setup and solve a Recurrence relation for the
number of calls made by F(n), the recursive algorithms for
‘computing nt
oR
‘Write an algorithm to find factorial using recursion. Find the
time complexity.
Soln, :
Factorial of number n is computed as : n
(n-2)%.t2"4
Recursive algorithm for finding factorial of any number
is described below:
*@-*
Algorithm FACTORIAL(n)
1 Description: Fi
A Anput: Integer vale n
i factorial of given number
number n
ifn ==Oor 1 then
return 1 @ Base case
else
retumn®(n=1) /Recursive case
end
For each call, problem size reduces by one and each call
performs one multiplication, We can setup the recurrence
as follow:
Cost of one
‘multiplication
‘T(n) = T(n- 1) +1 (Inductive case) (1)
Tn) =n, ifn=Oorn=
(Base case) \nalysis of Algorithms (MU)
hod
solves this recurrence using two meth
Forward ‘Substitution
From Equation (),
72)
T(3)
= Meterete?
= T(2jels24i=3
Alter ksteps
1) =k
Fork
Tn)
Backward substitution
To find ‘T(n), we need to
Tn D)stetus putin =n 1 in Equation (1),
= O(n)
Mmm 1) = Tn-1-1)41eT(n-2)41
So,T(n) = Tin 2)+141=1(n-2)42
Simitarly,
Tn-2) = Mn-2-1)612T(n-3)61
So, T(n) = Tn-3)+241=7(n~3)43
Aer k steps,
TO) = Tn
Fork = n,
Wek
Tn) = Tn-n)+n = 70) +n
4 Complexity Class
n= O(n)
Introduction to Complexity
. Explain the folowing:
(0) Computational comploxty
(t) Decision probioms
(Wi) Deterministic and ny
(W) Comploxty classes
(Y)_Intractabity,
Theory
'on-dlerministc algorithms
We can classify the
categorles :
2 The problem which can be
3 The problem which can
computationally they
takes ver
«I Dut can
"be solved,
n be solved
retlcally, buy
Algorithm,
™S, aNd the
rte fee tn
Force met
ind
|
(ane Practcaly no accep Forex
ehenemend of the gira by by
nd Tn = 1}, To solve
Introd
Del
Problem is called intractayy,,
ring,
takes too tong time 0 be solyeg 4” My
a
oh
robles has no practical appleny,
2 —
Problems which can be sojyy
: practically in a reasonable anon,
problems, there exists a determin” ;
that can solve the problem jn ¢ tay
p(n) is polynomial in, a pra
=.
Momign,
nd n is
Definition
If the problem is solvable in yyy,
ealled tractable. Such problerny 4), deny
|problems. ea
See _—<
5. The problem which is not known whey
not In P. Such problems falls. som,
class 3 and 4,
Definition
~The problem is said to be decision nk,
they produce output “Yes” or =
input. An algorithm which
problem is called decision a
> An optimization problem air
solution from the set of all feast
An algorithm whieh
Problem is called optimiz,
Optimization algorithm
or least cost solution
solve
solves op
‘ation algeria
seeks to find
Decision pro
‘onal problem can be functioa?®
© Such problem differs on 3
Classes are set of Leet
Complexity, like P_ probl
* Decision
te. The
melas falls 1
problems,
complexity of ar) F
iithin certain rans = Natingiatilily
Awaivels af Algunitnite (MU)
Problonne Which tale pr Vivwlly iinenapsta lili
Hind ww Pyne #0 Be ae are ed
jntrnwtable prablesiee
snninsny Heine Une ada té tin fa faired
fo Crp), when pride autthe polyianitind ii fh
ihre ob avptusonede Chee wise yf Che pin 1D
say that the phubleate han paokyeanitad ani
iy the
eamplenity.
Problent i vhwek the
aphictabonn nf anion, Sie iayiebility phiblewt
inne out whether fOr abe fMhpw aH sypuenaionh
fi toe fi Chir ervadginnonh
Redueibitity | het Bi aiid Beare tea prahtenna
and there evista: weinne abide iedonintio: algun (thin A
for Pr anet that Di ean be wolvedd in polynuniial
tine tasting Ac If the wane ahzoritiine eae aalbe
the problem Ps then uw ean eany thal Pe te
redueible ta Pic
1.4.2 P Probleme:
“@. What do you mean by P probleme’? Give an exainples
(6 Marke)
Q. Wile # short note on P probiens,
Definition
P problemn are net of probleme that ean be naliad:
in polynomial tine by determin iatte aigrithinnn
Pix abo known a8 PTIME or DTI voniplenity
alana, nia ah
They ave stnple to salve, enay ty verity anil take
computationally acceptable Unie far salvliyg any
sstanice: af the probleny, Stel prcbaleans ate alae Kinny
“tractable”,
Lu the worst ease, searching a element fran the Hato
ze 1 takes 1 compartaans, Phe number af eanapar tenn
ly with rospmet toy tnput nize, Su Hier
incre th
sare Ix P prabilen
Iv practice, most of the problem are 1 pralen
varelong, an elenvent tthe atvayy (EQ), tse ty ath
floment at the ened at Hike it (OHO) went slat
Uist selection ant(O(a’)), ling Heth ot Wee
COGioyen)), sort data Wednyg verge ant (AEHT0IN))
‘nate attention (a0) ae Faw a Che ewan of
problems,
‘alot with (26) cannon tos tote sane
eh gation
Wn tested on a prafiony at alae (0 1)
Ho not one tn ela Be
W vot tito all Ae
jvulytnntal eine
Hien Wa Analyte of Algor
ily
NMI OMAN FAKE He
isoprene HAW Fone appar cana
vtviad (pyaar abe
{oe eat ney magna bas sens valine sua fy
Hak (node palyninndal fie 968 fan, HE
Hivvend Ha aunt oluttine ies aah est PM, drape
‘vt ti ae UHH aii, Wine te ate
laut
Fevamplow of F yablan
1
d
4
‘
i
1
{a
Avon HH av
Merwe ail
Hive outed
‘Manna anntijilt ations
Jilin. ontitinini and nian eleriont th
NP Problem
Wivachiileonit (OSI aTENEnTTS
HWP fe aut of yraitons whieh can be solved tn non
lujorsitie joalynunatal tie, AP oes not mean non
juwlyninital, Waban tay ou Deter naiatte Polynonntal
tine
‘Hh nun-aetoyministhe alyorittin operates In two
lane
Nondotorminiatio (quessiag) stage + flor input
Hivstanee 1, anne anlution string 8 ts generated, which
‘oa he tht tn eatcate alt
Dotwrmbulatie (verification) stage 11 and 8 are given
fae ane Tnjul te) thee slaternitndstte algorithm, whieh
folie "Wee (Fb soution for input instance f
foliation (1 NP problems cannot te obtatned tn
aly fine, hut glven the solution, 1 ean be
\vetoa tn pyle
NP ne hilo all praitonns oF Py be PND
Ninapnarh prablon (02), Peavelling salesman
Jnvahdony ({n1)}, Pawar at Hawt (O(2"= 1D),
Hanutliontin eyete (OQul)) are exanyples of NP
often,
norma ave further classified tn NP-complete and
Hig, WALL shows the taxonomy of
aval eatengon
nip ytiy chase
Proviens |
iin]
Fig. 1.4.1 | Taxonomy of complexity clas NE Analysis of Algorithms (MU)
1.4.4. Difference between P and NP
Q. Differentiate between P and NP. mn
NP Problem
Sr,
P Problem
Aa ee vaya gatos toch
stands for Stands for Non-
polynomial Deterministic Polynomial
Problem can not be solved
in polynomial time, but
piven the solution it can be
verified in polynomial time.
Problem be
solved in polynomial
time,
can
3
NP prablems are superset
P problems are sub
set of NP problems.
All P- problems are
deterministic in
ature,
AIL NP problems are non-
leterministic in nature.
Ex: Selection
Binary search,
Sort, | Ex: Knapsack problem, TSI
problem,
1.4.5 NP-Completeness and Reducibility
Q. What is Reduction in NP-complotenoss proofs? Whal
‘are types of reductions? (7 Marks)
Q, Explain Polynomial Timo Algor (6 Marks)
4-18.
Inroducton 1 Anja
Sa,
Definition
he potmomial time reduction
solving problem A by the hypothtigg,
rolving aigferent problem B,
polynomial time.
Basically, the polynomial reduction jg.
showing that the problem Ais 01 hare
problem B. |
= For example, we ave some hypoth
which can sort numerical data in some pons"
‘The Input tothe algorithm can Only be i ney
Suppose we have a new problem t0 sort ping,
cities across the country. What can be done?
= Suppose we don't have an efficient algorithm ig,
string data, We can apply some hashing function,
names to map them to numeric Values, Noy,
"| identical to the first approach.
= Thus, the polynomial reduction is the way of uni,
problem into another problem whose solution
found in polynomial time,
= Reduction takes one of the three forms
1, Restriction
2. Local Replacement
3. Component design
applying some transformation function on x
So given an input x to A, reduction algorithm produces
“yes” only input x to A returns "Yes
Input x tor
‘oclon
109) tho
probiom A,
Input to 8
Tranetorm
Algorithm tor A
Fig.
2
hus, we ay As yaynom ne edule to th
input x
© produced for A,
© f(x) should be computable in poh
reduetble to B, di
Ber.
(© Transformation function f{.) may
as x would
nsw
Iynomial time
wted aS AHHH ap
Consider two declston problems A and B, Reduction from A
Intermediate
if 14.2 shows the scenario,
ents
oF probley
of x,
Be P they
‘o Biransforms input x of A to equivalent input fel
result {(x) to problem B such that f(x) to Bt
Yos/No | Yos/ No
oulput for B output for A
Agorhav aD) On mput x) Fon input
+ >
Reduction
some
“ransformation function () such that
m Ato o
'0 109) such that, input (x) to B produce th
Hsueh fi
Unction ¢
NAG p,
al
fats, we say A is polyol
However this does not imply #°!
we
| 1-19
Introduction to Analysis of Algorith
Anaysis of Aigorthms (MU)
shows the mode of reduction from one
— Rg 143
problem to other.
Eaypenne
Out SAT
OF SAT
scl elaseent
asi
‘canserar do8on
sclrelscerent Local epiacement | Component decgn Camgenentesgn
Pee
wow] [Seen] [Base
Sse) bs 8
[Resicion' ——PRastcton
Resse Ter.
Fig, 1.4.3 : Reductions used in some fundamental
NNP-completeness proof
1.4.6 NP-Completeness Proofs
@. What do you mean by NP-Compiete Problems? Give
an example. (2 Marks)
‘Q. What are the conditions to prove that a problem P is
NP-Complete? (8 Marks)
@. Write short notes on steps for NP Completeness
proofs. Een
1fB< >A, implies B is reducible to A and B is not harder
than A by some polynomial factor.
Definition
Decision problem C is called NP-complete if it
has following two properties :
1.Cis in NP, and
2Every problem X in NP is reducible to C in
polynomial time, i.e. For every X € NP, X $C.
‘These two facts prove that NP-complete problems are
the harder problems in class NP. They are often referred
as NPC.
Problem satisfying condition 2 is sald to be NP-hard,
whether or not it satisfies condition 1.
If any NP-complete problem belongs to class P, then
P=NP.
However, a solution of any NP-complete problem can be
verified in polynomial time, it cannot be obtained in
polynomial time.
Method for solving NP-complete problems in reasonable
time remains undiscovered. NP-complete problems are
often solved using randomization algorithms, heuristic
approach or approximation algorithms.
Examples of NP-Complete problems
y
2
3)
4)
5)
6)
7
8) Clique problem.
1.4.6(A) Vertex Cover Problem
ee
@. Specify one example of the NP-complete problem.
‘Also, ustty that why itis NP-complete. (10 Marks)
. Prove that vertex cover problem is NP complete.
(Eee eee
Boolean satisfiability problem.
Knapsack problem
Hamiltonian path problem.
‘Traveling salesman problem.
Subset sum problem.
Vertex cover problem.
Graph colouring problem.
Definition
Vertex cover of Graph G = (V, B) is set of vertices
such that any edge (u, v) € B, incident to at least
one vertex in the cover. In other words, vertex cover
is a subset of vertices Vc V such that if the edge
(u, ») € Ethenu € Vorv eV.
‘The size of the cover is a number of vertices in Ww
Vertex cover problem is to find out such
minimum size cover. A decision problem is to
check if given graph has vertex cover of size k.
= The simplest way of finding vertex cover of graph
G= (V, E)is to randomly select the edge (u, v) € E and
delete adjacent edges of u and v.
— Repeat the procedure until the cover is found. For
example, consider Fig. 144,
Fig. 1.44
Wrest sree
“2
Seer tn ey ER
Ca (ab 6) form aang wr
Let S represents the solution set. inti
.2> a8
SREP 1: Select any random edge, let us select edge <1
shown inthe Pg 1.45. Remove the incident oder
of vertex 1 and 2 So, $* (1.2)
Fig. 14.5: Atter selecting edge <1. 2>
‘Stil few edges are not adjacent to vertices in S, $0 60 08.
Step 2: Let us now select edge <5, 6» Remove adjacent
san
© 6»
SaS= (1,
Fig. 1.46 : After selecting edge <5, 6>
Stil few edges are not adjacent to vertices in S10 g0 0m.
‘Step 3: Let us now select the edge <3, 7»,
SoS = (1,2, 3.5.6) and all the edges are adjacent to at
least cne vertex is S.SoS isthe cover of graph G.
o—-®
~ However, Sts the cover ofthe graph but it may not be
fminimsim Instead of selecting edge <5, 6» instep 2
wwe would have selected edge <3. 6>, it would have
resulted in a minimum numberof vertices
Theorem : Vertex cover is NP-complete
Proof
- Fe prove that varies cover is NP-complete, se wy
inate USAT Probes to vertex cover problem. Lat ¢ pe
the Boolean function with k causes
~ For each Weral an’ in the cause, we create
an
vw tn Fig, 148 Lage is truth vetting coment
‘ertex cover must include at eas one of or 3
Fig. 140
= Vertex caer of such graph conta:
~ Let os build the graph 3-SAT Bootes
Fig. 149
= hay verte cover wil ave 1 chute 2 ae
vertices trom (a,b. 6. Join the correspensy,
from triangle to edge a: per cise
where 9 is a pumber of variables
clauses
ota
‘This Boolean expression generates the grat x,
in Pig 14.10
bre) arbedirced
Fig. 14.10
For given Boolean function,
4» number of variables © ¢
number of clauses «3
kensameto
$2 vertex cover of this graph must costae 1°
One verten of each trungie and one vets "4
ope
oo
of vertex cover {Wants o Algortms (MU)
Fig. 4.11
41.4.6(B) Clique Problem
@, Prove that Clique Decision Problem is NP-Hard,
(7 Marks)
Q. Prove that a clique problem is NP-complete
(7 Marks)
Problem
= Clique is the complete subgraph of graph G. In complete
subgraph, there exists an edge between every pair of
vertices.
The size of a clique is given by a number of vertices init.
Max clique is the clique of maximum size.
Finding max clique is obviously optimization problem.
Checking if graph G has a clique of size k is decision
problem,
‘The optimization problem is to find a clique of
‘maximum size for given graph. A decision problem is to
check whether a clique of size k exists for graph G.
Q—O
SZ
YS
‘Graph aque of ze 3 chquo taxes
Fig, 1.4.12 : Graph G and its cliques
Let algorithm CLIQUE(G, k) returns true if graph G has a
clique of size k. We shall start with k = n,n -
.-until CLIQUE(G, k) returns true,
Theorem : Clique Decision Problem is NP-
complete
~ To prove that clique belongs to NP-complete, we use V"
as a certificate for graph G. Checking ifV'is a clique, can
be done in polynomial time by checking the presence of
edge for each u,veV".
‘To show that clique is NP-complete, we will show that 2-
CNF-SAT S, CLIQUE,
{ntroduction to Analysis of Algority
= Let = CLA C2A CA 4G bea Boolean function of k
‘clause, where each clause C, 1s in 3-CNE, Le. each clause
has exactly three literals. We shall construct a graph
such that Boolean function @ be satisfiable i and
Ghas a clique of size k. The graph can be constr
follows:
Each vertex corresponds toa litera
= Connect each vertex to remaining all vertices in
remaining clause except for and —x
Example: 02 (s:V4¥V 735) A( ANY VIA) A GLY
v8)
We shall show that the transformation @ to G is
polynomial reduction. Let us consider that 9 has
satisfying assignment, All literals x: in each clause are
ORed with each other.
= Soat least one literal in each clause is assigned value 1
If we pick up one such literal from each clause, it forms
set V' of k vertices as we have k clauses in 9. There exist
an edge for each u, veV’
Fig. 1.4.13
NF satisfiability is NP-complete and it is reducible to
clique, so clique is also NP-complete.
1.4.7. NP-Hard Problem
@. Explain class : NP-Hard,
Formally, a decision problem p is called NP-hard,
every problem in NP can be reduced to p in polynomial
time.
= NP-hard is a superset of all problems, NPC is in
NP-hard, but converse may not be true.
NP-hard problems are at least as hard as the hardest
(2 Marks)
problems in NP. lysis of Algorithms (MU)
7
om
1rd problem in polynomia,
Meee problems in NP
if we can solve a
tine, we would beable save all te
in polynomial time.
~ NP-hard problems do not have to be in
may not be decision problem. F
~ Subset subproblem, travelling salesman problem "8
NPC and also belong to NP-hard. There are ce
i they are not
problems which belong to NP-hard but they
NP-complete. :
- A well-known example of the NP-hard problem Is
Halting problem,
~The halting problem is stated as, “Given an algorithm
and set of inputs, will it run forever The answer to
this question is Yes or No, so this is decision problem.
~ There does not exist any known algorithm which can
decide the answer for any given input in polynomial
‘me, So halting problem is NP-hard problem.
~ Different mathematicians have given different
relationship considering possibilities of P = NP and
PaNp.
~The relationship between P, NP, NP-Complete and NP-
hard is described in below Fig. 1.4.14 assuming that
P # NP. This is widely accepted relationship among
these complexity classes.
‘NP complete
J
Fig. 1.4.14
Ip. Even they
lem are
NP
1.4.8 Comparison between NP Hard and
NP Complete
©. Ditferentiato between NP-hard and NPeompisi
algorithms, (4 Marks)
Is Problem y
Problem —X ~
complete Hf NP problem! NP-compjyg My
¥ is reducible to X infreducihie
polynomial time,
—
1s, | Ex.3 SAT problem. Hal ote
mt
1.5 Analysis of Selection Sor ang
Insertion Sort
___Insertion Sort
4.5.1 Selection Sort
@, Explain selection sort and drive its conpiga
(oy
ky
Like bubble sort, selection sorts also compan,
in place algorithm. Selection sort is simple ayy
obvious advantage of minimum swaps any
algorithms. For list of size n, selection sor ps
maximum (n ~ 1) swaps. However,
quadratic and hence not accepted for a large list
Funing
~ In every iteration 4, Selection sort finds the mies
element from the unsorted list and swap it with:
element in the list. At the end of the i pass, i dene
Bet sorted. Sorting starts from the beginning
~ At the end of the first pass, minimum element sex:
the first location, in the second pass, second misc:
clement seats on the second location and so on.
~ Wf the list is Teverse sorted, bubble sort does n- 1s
in the first pass, whereas selection sort does ea
