Noida Institute of Engineering and Technology, Greater

Noida

Introduction to Algorithm
and Asymptotic Notations

Unit: 1

Design & Analysis of Algorithms

Sachin Singh
Assistant Professor
CS(AI)

Sachin Singh DAA Unit I

1
June 13, 2022
 Brief Introduction About Me

Name : Sachin Singh

Designation :Assistant Professor CS(AI) Department

NIET Greater Noida

Qualification:

 B.Tech AKGEC Ghaziabad in 2017.

 M.Tech Delhi Technological University(formely DCE) in 2020.

June 13, 2022 Sachin Singh DAA Unit I 2

 Evaluation Scheme

June 13, 2022 Sachin Singh DAA Unit I 3

 Syllabus

Sachin Singh DAA Unit I

June 13, 2022 4
 Branch wise Application

• First, we will start with the internet which is very much important for our daily life and we
cannot even imagine our life without the internet and it is the outcome of clever and creative
algorithms. Numerous sites on the internet can operate and falsify this huge number of data
only with the help of these algorithms.

• The everyday electronic commerce activities are massively subject to our data, for example,
credit or debit card numbers, passwords, OTPs, and many more. The centre technologies
used incorporate public-key cryptocurrency and digital signatures which depend on
mathematical algorithms.

• Even an application that doesn't need algorithm content at the application level depends
vigorously on the algorithm as the application relies upon hardware, GUI, networking, or
object direction and all of these create a substantial use of algorithms.

• There are some other vital use cases where the algorithm has been used such as if we watch
any video on YouTube then next time we will get related-type advice as recommended videos
for us.

June 13, 2022 Sachin Singh DAA Unit I 5

 Course Objective

• Upon completion of this course, students will be able to do the

following:
• Analyze the asymptotic performance of algorithms.
• Write rigorous correctness proofs for algorithms.
• Demonstrate a familiarity with major algorithms and data structures.
• Apply important algorithmic design paradigms and methods of
analysis.
• Synthesize efficient algorithms in common engineering design
situations.

Sachin Singh DAA Unit I

June 13, 2022 6
 Course Outcome

At the end of the semester, the student will be able:

Description Bloom’s Taxonomy
CO To have knowledge of basic principles of algorithm design and Knowledge, analysis
1 Analysis, asymptotic notations and growth of functions for time And design
and space complexity analysis and applying the same in different
sorting algorithms
CO To apply different problem-solving approaches for advanced Knowledge, analysis
2 data structures And apply
CO To apply divide and conquer method for solving merge sort, Knowledge, analysis and
3 quick sort, matrix multiplication and Greedy Algorithm for Apply
solving different Graph Problem.

CO To analyze and apply different optimization techniques like Knowledge, Analysis And
4 dynamic programming, backtracking and Branch & Bound to Apply
solve the complex problems
CO To understand the advanced concepts like NP Completeness and Knowledge, Analysis and
5 Fast Fourier Transform, to analyze and apply String Matching, Apply
Approximation and Randomized Algorithms to solve the
complex problems

June 13, 2022 Sachin Singh DAA Unit I 7

 Program Outcome

At the end of the semester, the student will be able:

POs Engineering Graduates will be able to
PO1 Engineering Knowledge
PO2 Problem Analysis
PO3 Design & Development of solutions
PO4 Conduct Investigation of complex problems
PO5 Modern Tool Usage
PO6 The Engineer and Society
PO7 Environment and sustainability
PO8 Ethics
PO9 Individual & Team work
PO10 Communication
PO11 Project management and Finance
PO12 Life Long Learning

June 13, 2022 Sachin Singh DAA Unit I 8

 CO-PO and PSO Mapping

Design and Analysis of Algorithm (kCS-502)

PO PO1
CO.K PO2 PO3 PO4 PO5 PO6 PO7 PO8 PO9 PO10 PO12
1 1

ACSE0401.1 3 3 3 3 2 - - - 2 2 - 3

ACSE0401.2 3 3 3 3 2 2 - 1 1 1 - 3

ACSE0401.3 3 3 2 3 3 2 - 2 1 1 2 3

ACSE0401.4 3 3 3 3 2 2 - 2 2 1 3 3

ACSE0401.5 2 2 2 2 2 2 - 2 1 1 1 2

Average 2.8 2.8 2.6 2.8 2.2 1.6 - 1.8 1.4 1.2 1.2 2.8

June 13, 2022 Sachin Singh DAA Unit I 9

 Program Educational Objectives(PEOs)

PEO1: To have an excellent scientific and engineering breadth so as to

comprehend, analyze, design and provide sustainable solutions for
real-life problems using state-of-the-art technologies.
PEO2:To have a successful career in industries, to pursue higher studies or
to support enterpreneurial endeavors and to face global challenges.
PEO3:To have an effective communication skills, professional attitude,
ethical values and a desire to learn specific knowledge in emerging
trends, technologies for research, innovation and product
development and contribution to society.
PEO4: To have life-long learning for up-skilling and re-skilling for
successful professional career as engineer, scientist, enterpreneur
and bureaucrat for betterment of society

June 13, 2022 Sachin Singh DAA Unit I 10

 Result Analysis

June 13, 2022 Sachin Singh DAA Unit I 11

 End Semester Question Paper Template

B TECH
(SEM-V) THEORY EXAMINATION 20__-20__
DESIGN AND ANALYSIS OF ALGORITHMS
Time: 3 Hours Total
Marks: 100
Note: 1. Attempt all Sections. If require any missing data; then choose
suitably.
SECTION A
1.Q.No.
Attempt all questions in brief.
Question Marks 2 xCO10 =
20
1 2
2 2
. .
10 2

June 13, 2022 Sachin Singh DAA Unit I 12

 End Semester Question Paper Templates

SECTION B
2. Attempt any three of the following: 3 x 10 = 30

Q.No. Question Mark CO

s
1 10
2 10
. .
5 SECTION C 10
3. Attempt any one part of the following: 1 x 10 = 10

Q.No. Question Marks CO

1 10
2 10
June 13, 2022 Sachin Singh DAA Unit I 13
 End Semester Question Paper Templates
4. Attempt any one part of the following: 1 x 10 = 10
Q.No. Question Marks CO

1 10
2 10
5. Attempt any one part of the following: 1 x 10 = 10
Q.No. Question Marks CO
1 10
2 10

6. Attempt any one part of the following: 1 x 10 = 10

Q.No. Question Mark CO
s

1 10
2 10

June 13, 2022 Sachin Singh DAA Unit I 14

 Prerequisite and Recap

• Prerequisite
• Basic concept of c programming language.
• Concept of stack, queue and link list.

• Recap
• Flow Chart
• Algorithm

June 13, 2022 Sachin Singh DAA Unit I 15

 Introduction About Subject With Videos

• The word Algorithm means “a process or set of rules to be followed in calculations

or other problem-solving operations”. Therefore Algorithm refers to a set of
rules/instructions that step-by-step define how a work is to be executed upon in
order to get the expected results.

• https://youtu.be/6hfOvs8pY1k

June 13, 2022 Sachin Singh DAA Unit I 16

 Unit Content

• Introduction to Algorithm
• Characteristics of Algorithm
.
• Analysis of Algorithm
• Asymptotic Notations
• Recurrence Relation
• Sorting and order Statistics
• Shell sort,
• Heap sort,
• Sorting in linear time

June 13, 2022 Sachin Singh DAA Unit I 17

 Unit Objective

.
• This is an introductory chapter of Design & Analysis of Algorithm
covering the concept, importance and characteristics of algorithms. The
complexity and its calculation has been explained. Further, recursion and
different methodologies to solve them have also been provided.
• In other part of unit different sorting algorithm – Insertion, Heap sort,
Quick sort, bucket sort, Radix Sort and Counting sort have been discussed
along with their complexities

June 13, 2022 Sachin Singh DAA Unit I 18

 Topic Objective

• The objective of this topic is to make students understand

about
• Need of algorithm
• Problem statement
• Why study algorithm
• Characteristics of Algorithm

June 13, 2022 Sachin Singh DAA Unit I 19

 Introduction to the Algorithm(CO1)

An algorithm provides a step-by-step method for solving a

computational problem, algorithms are not dependent on a particular

programming language, machine, system, or compiler.

Problem

Algorithm

Computer
Input output

June 13, 2022 Sachin Singh DAA Unit I 20

 Algorithm

The Study of algorithm is necessary from the following point of view.

• Algorithm help us to understand scalability.

• Efficient algorithm leads to efficient program.

• Efficient program sell better.

• Efficient program make better use of hardware.

June 13, 2022 Sachin Singh DAA Unit I 21

 Characteristics of Algorithm

• Correctness

• Efficiency

• Simplicity

• Generality

• Non Ambiguity

June 13, 2022 Sachin Singh DAA Unit I 22

 Algorithm

All algorithms must satisfy following criteria-:

– Input
– Output
– Definiteness
– Finiteness
– Effectiveness

June 13, 2022 Sachin Singh DAA Unit I 23

 Algorithm

• An algorithm is a set of steps of operations to solve a problem.

• An algorithm is an efficient method that can be expressed within

finite amount of time and space.

• Algorithm is independent from any programming languages.

• Algorithm design include creating an efficient algorithm to solve a

problem in an efficient way using minimum time and space.

• Time is directly proportional to the number of operations on a

program.

June 13, 2022 Sachin Singh DAA Unit I 24

 Analysis of Algorithm(CO1)- Objective

The objective of this topic is to make students understand about

• Space Complexity
• Time Complexity
• Best Case
• Worst Case
• Average Case

June 13, 2022 Sachin Singh DAA Unit I 25

 Prerequisite and Recap

• Prerequisite
• Definition of algorithm

• Recap
• Characteristics of algorithm

June 13, 2022 Sachin Singh DAA Unit I 26

 Analysis of Algorithm(CO1)

– The term "analysis of algorithms" was coined by Donald Knuth.

– To estimate the complexity function for arbitrarily large input.

– Analysis of algorithms is the determination of the amount of time

and space resources required to execute it.

– The efficiency or running time of an algorithm is stated as a function

relating the input length to the number of steps, known as time
complexity, or volume of memory, known as space complexity.

– The main concern of analysis of algorithms is the required time or

performance

June 13, 2022 Sachin Singh DAA Unit I 27

 Analysis of Algorithm(CO1)

How to measure time

Count total number of fundamental operation in program and this

total will give the rough estimate of the running time in terms of
input size.

• Best Case
The minimum number of steps taken on any instance of size n.

• Average Case
An average number of steps taken on any instance of size n.

• Worst Case
The maximum numberSachin
June 13, 2022
of steps
Singh
taken
DAA
on any
Unit I
instance of size n.28
 Asymptotic Notations(CO1)- Objective

The objective of this topic is to make students understand about

• Five Asymptotic notations

 Big Theta Notation
 Big Oh Notation
 Big Omega Notation
 Small Oh Notation
 Small Omega Notation

June 13, 2022 Sachin Singh DAA Unit I 29

 Prerequisite and Recap

• Prerequisite
• Definition of algorithm

• Recap
• Cases of analysis of algorithm

June 13, 2022 Sachin Singh DAA Unit I 30

 Asymptotic Notations (CO1)

The asymptotic running time of an algorithm are defined in terms of

functions whose domains are the set of natural numbers. The
following are the types of asymptotic notation

 Big Theta Notation

 Big Oh Notation
 Big Omega Notation
 Small Oh Notation
 Small Omega Notation

June 13, 2022 Sachin Singh DAA Unit I 31

 Big Theta Notation

The function f(n) = Θ(g(n)) (read as “ f of n is theta of g of n”) if there

exist positive constants c1 and c2 and n0 such that
c1g(n)≤f(n)≤c2g(n) for all n≥ n0

June 13, 2022 Sachin Singh DAA Unit I 32

 Big Theta Notation

Example

Let us consider a given function,

f(n)=4.n3+10.n2+5.n+1

Considering g(n)=n3,

4.g(n)⩽f(n)⩽5.g(n) for all the large values of n.

Hence, the complexity of f(n) can be represented as

θ(g(n)), i.e. θ(n3)

June 13, 2022 Sachin Singh DAA Unit I 33

 Big Oh Notation

The function f(n) = O(g(n)) (read as “ f of n is big oh of g of n”) if there

exist positive constants c and n0 such that
f(n)≤cg(n) for all n≥ n0

June 13, 2022 Sachin Singh DAA Unit I 34

 Big Oh Notation

Example

Let us consider a given function,

f(n)=4.n3+10.n2+5.n+1

Considering g(n)=n3,

f(n)⩽5.g(n) for all the values of n>2.

Hence, the complexity of f(n) can be represented as

O(g(n)), i.e. O(n3)

June 13, 2022 Sachin Singh DAA Unit I 35

 Big Omega Notation

The function f(n) = Ω(g(n)) (read as “ f of n is Ω of g of n”) if there

exist positive constants c and n0 such that
cg(n)≤ f(n) for all n≥ n0

June 13, 2022 Sachin Singh DAA Unit I 36

 Big Omega Notation

Example

Let us consider a given function,

f(n)=4.n3+10.n2+5.n+1

Considering g(n)=n3,

f(n) ≥ 4.g(n) for all the values of n>0.

Hence, the complexity of f(n) can be represented as

Ω (g(n)), i.e. Ω(n3)

June 13, 2022 Sachin Singh DAA Unit I 37

 Small Oh Notation

• Upper bound that is not asymptotically tight

f(n) = o(g(n)) if f(n)< c(g(n)) for all n≥ n0

• f(n) becomes insignificant relative to g(n) as n approaches infinity:

lim [f(n) / g(n)] = 0
n

• g(n) is an upper bound for f(n) that is not asymptotically tight.

June 13, 2022 Sachin Singh DAA Unit I 38

 Small Oh Notation

Example

Let us consider a given function,

f(n)=4.n3+10.n2+5.n+1

Considering g(n)=n4,
lim (4.n3+10.n2+5.n+1/ n4 )=0
n→∞

Hence, the complexity of f(n) can be represented as

o(g(n)), i.e. o(n4)

June 13, 2022 Sachin Singh DAA Unit I 39

 Small Omega Notation

• lower bound that is not asymptotically tight

f(n) = ω(g(n)) if c(g(n))< f(n) for all n≥ n0

• f(n) becomes arbitrarily large relative to g(n) as n approaches

infinity:
lim [f(n) / g(n)] = .
n

• g(n) is a lower bound for f(n) that is not asymptotically tight.

June 13, 2022 Sachin Singh DAA Unit I 40

 Small Omega Notation
Example

Let us consider a given function,

f(n)=4.n3+10.n2+5.n+1

Considering g(n)=n2,
lim (4.n3+10.n2+5.n+1/ n2 )=0
n→∞

Hence, the complexity of f(n) can be represented as

ω (g(n)), i.e. ω(n4)

June 13, 2022 Sachin Singh DAA Unit I 41

 Recursion (CO1)- Objective

The objective of this topic is to make students understand about

• Different methods of solving recursion

• Substitution/ Iteration method

• Recursion Tree method

• Master method

June 13, 2022 Sachin Singh DAA Unit I 42

 Prerequisite and Recap

• Prerequisite
• Algorithm
• Mathematical concepts of limits and summation

• Recap
• Asymptotic notations

June 13, 2022 Sachin Singh DAA Unit I 43

 Recursion (CO1)
• The process in which a function calls itself directly or indirectly is
called recursion and the corresponding function is called as
recursive function.
• Using recursive algorithm, certain problems can be solved quite
easily.
• Examples of such problems are Towers of Hanoi (TOH), Inorder/
Preorder/Postorder Tree Traversals, DFS of Graph, etc.
• The running time of recursive algorithms is represented by an
equation that describes a function in terms of its value on smaller
functions. For eg.

T(n) = {
C

2T(n/2) +cn
n=1

n>1

June 13, 2022 Sachin Singh DAA Unit I 44

 Recursion

There are three methods to solve recurrence

• Substitution/ Iteration method

• Recursion Tree method

• Master method

June 13, 2022 Sachin Singh DAA Unit I 45

 Recursion

Substitution/ Iteration method

Guess the form of the answer by experience and creativity or by some

heuristics guess the form of the solution. And then use induction to
find the constants and show that solution works.

Example Solve T(n)=T(n-1) + 1 and T(1) = Ө(1)

T(n) = T(n-2) +1+1
T(n) = T(n-3) +1+1+1
T(n) = T(n-4) +1+1+1+1
And so on T(n) = T(n-k) + k
Where n-k =1 => n-k=1
Thus T(n)= T(1) + n-1⇒Ө(n)

June 13, 2022 Sachin Singh DAA Unit I 46

 Recursion

Recursion Tree Method

Each node represents the cost of a single sub problem.

• Sum up the costs with each level to get level cost.
• Sum up all the level costs to get total cost.
• Particularly suitable for divide-and-conquer recurrence.
• Best used to generate a good guess, tolerating “sloppiness”.
• If trying carefully to draw the recursion-tree and compute cost, then
used as direct proof.

Example Recursion Tree ofT(n)=T(n/3)+ T(2n/3)+O(n)

June 13, 2022 Sachin Singh DAA Unit I 47

 Recursion

June 13, 2022 Sachin Singh DAA Unit I 48

 Recursion

Master method

• Master Method is a direct way to get the solution.

• The master method applies to recurrences of the form

T(n) = a T(n/b) + f (n) ,
where a ³ 1, b > 1, and f is asymptotically positive.

June 13, 2022 Sachin Singh DAA Unit I 49

 Recursion

• There are three cases:

• Case I f (n) = O(nlogba – e) for some constant e > 0.
f (n) grows polynomially slower than nlogba (by an ne factor).
Solution: T(n) = Q(nlogba)

Example
. T(n) = 4T(n/2) + n
Here a = 4, b = 2  nlogba = n2; f (n) = n.
CASE 1: f (n) = O(n2 – e) for e = 1.
 T(n) = Q(n2).

June 13, 2022 Sachin Singh DAA Unit I 50

 Recursion

• Case II f (n) = Q(nlogba lgkn) for some constant k ³ 0.

f (n) and nlogba grow at similar rates.
Solution: T(n) = Q(nlogba lgk+1n) .

Example

T(n) = 4T(n/2) + n2
Here a = 4, b = 2  nlogba = n2; f (n) = n2.
CASE 2: f (n) = Q(n2lg0n), that is, k = 0.
 T(n) = Q(n2lg n).

June 13, 2022 Sachin Singh DAA Unit I 51

 Recursion

Case III f (n) = W(nlogba + e) for some constant e > 0.

f (n) grows polynomials faster than nlogba (by an ne factor),
and f (n) satisfies the regularity condition that a f (n/b) £ c f (n)
for some constant c < 1.
Solution: T(n) = Q( f (n) ) .

Example
T(n) = 4T(n/2) + n3
a = 4, b = 2  nlogba = n2; f (n) = n3.
CASE 3: f (n) = W(n2 + e) for e = 1 and 4(cn/2)3 £ cn3 for c =
1/2.
 T(n) = Q(n3).

June 13, 2022 Sachin Singh DAA Unit I 52

 Recursion

Examples of some standard algorithms whose time complexity

can be evaluated using Master Method

Merge Sort: T(n) = 2T(n/2) + Θ(n). It falls in case 2 as c is 1 and Log ba]
is also 1. So the solution is Θ(n Logn)

Binary Search: T(n) = T(n/2) + Θ(1). It also falls in case 2 as c is 0 and

Logba is also 0. So the solution is Θ(Logn)

June 13, 2022 Sachin Singh DAA Unit I 53

 Sorting(CO1)- Objective

The objective of this topic is to make students understand about

• Different Sorting Tehniques

• Insertion sorting
• Heap Sort
• Counting Sort
• Bucket Sort
• Heap Sort

June 13, 2022 Sachin Singh DAA Unit I 54

 Prerequisite and Recap

• Prerequisite
• Algorithm
• loops- for while
• C programming

• Recap
• Recursion

June 13, 2022 Sachin Singh DAA Unit I 55

 Sorting(CO1)

Insertion Sort

• It inserts each element of the array into its proper position, leaving
progressively larger stretches of the array sorted.

• This sorting iterates down an array, and the part of the array already
covered is in order; then, the current element of the array is inserted
into the proper position at the head of the array, and the rest of the
elements are moved down, using the space just vacated by the element
inserted as the final space.

June 13, 2022 Sachin Singh DAA Unit I 56

 Sorting(CO1)

Insertion Sort Algorithm

Algorithm: We use a procedure INSERTION_SORT. It takes as

parameters an array A[1.. n] and the length n of the array.

INSERTION_SORT (A)

1. FOR j ← 2 TO length[A]
2. DO key ← A[j]
3. //{Put A[j] into the sorted sequence A[1 . . j − 1]}
4. i ← j − 1
5. WHILE i > 0 and A[i] > key
6. DO A[i +1] ← A[i]
7. i←i−1
8. A[i + 1] ← key
June 13, 2022 Sachin Singh DAA Unit I 57
 Sorting(CO1)

June 13, 2022 Sachin Singh DAA Unit I 58

 Sorting(CO1)
Insertion Sort

Three Cases:
Best-Case The best case occurs if the array is already sorted. the
while-loop in line 5 executed only once for each j. This happens if
given array A is already sorted. T(n) = an + b = O(n) It is a linear
function of n.

Worst case The worst-case occurs if the array is sorted in reverse

order i.e., in decreasing order. the worst-case occurs, when line 5
executed j times for each j. This can happens if array A starts out in
reverse order T(n) = an2 + bn + c = O(n2)

Average case: When half of the array is sorted and half of the array is
unsorted it produces O(n2)

June 13, 2022 Sachin Singh DAA Unit I 59

 Sorting(CO1)

Heap Sort

• Heap sort is a comparison based sorting technique based on Binary

Heap data structure.

• The binary heap data structure is an array that can be viewed as

almost complete binary tree.

• Each node of the binary tree corresponds to an element of the array.

• It is similar to selection sort where we first find the maximum

element and place the maximum element at the end.

• We repeat the same process for remaining element.

June 13, 2022 Sachin Singh DAA Unit I 60

 Sorting(CO1)

Heap Sort
• For Example
• We represent heaps in level order, going from left to right.

• The array corresponding to the heap above is [25, 13, 17, 5, 8, 3].

June 13, 2022 Sachin Singh DAA Unit I 61

 Sorting(CO1)

Heap Sort
The root of the tree A[1] and given index i of a node, the indices of its
parent, left child and right child can be computed as

PARENT (i) = return floor(i/2)

LEFT (i) = return 2i
RIGHT (i) = return 2i + 1

June 13, 2022 Sachin Singh DAA Unit I 62

 Sorting(CO1)
Heap Property
1. Max-heap property: A max-heap is a binary tree such that. - the
data contained in each node is greater than (or equal to) the data in
that node's children. ie. for every node i other than the root
A[PARENT(i)] >= A[i]

June 13, 2022 Sachin Singh DAA Unit I 63

 Sorting(CO1)
2. Min-heap property: A min-heap is a binary tree such that. - the
data contained in each node is less than (or equal to) the data in that
node's children. - the binary tree is complete
for every node i other than the root
A[PARENT(i)] <= A[i]

June 13, 2022 Sachin Singh DAA Unit I 64

 Sorting(CO1)

Four basic procedures on heap are

1. Heapify, which runs in O(lg n) time.

2. Build-Heap, which runs in linear time.

3. Heap Sort, which runs in O(n lg n) time.

4. Extract-Max, which runs in O(lg n) time.

June 13, 2022 Sachin Singh DAA Unit I 65

 Sorting(CO1)

Heapify (A, i)

1. l ← left [i]
2. r ← right [i]
3. if l ≤ heap-size [A] OR A[l] > A[i]
4. then largest ← l
5. else largest ← i
6. if r ≤ heap-size [A] OR A[r] > A[largest]
7. then largest ← r
8. if largest ≠ i
9. then exchange A[i] ↔ A[largest]
10. Heapify (A, largest)

June 13, 2022 Sachin Singh DAA Unit I 66

 Sorting(CO1)

Build Max Heap (A)

1. heap-size [A] ← length [A]

2. For i ← lower bound (length [A])/2 down to 1
3. Do Max Heapify [A, i]

June 13, 2022 Sachin Singh DAA Unit I 67

 Sorting(CO1)
Characteristics of Heap Sort Algorithm

• The heap sort combines the best of both merge sort and insertion
sort. Like merge sort, the worst case time of heap sort is O(n log n) and
like insertion sort, heap sort sorts in-place.

• The heap sort algorithm starts by using procedure BUILD-HEAP to build

a heap on the input array A[1 . . n].

• Since the maximum element of the array stored at the root A[1], it can
be put into its correct final position by exchanging it with A[n] (the last
element in A).

• If we now discard node n from the heap than the remaining elements
can be made into heap. Note that the new element at the root may
violate the heap property.
June 13, 2022 Sachin Singh DAA Unit I 68
 Sorting(CO1)

Heap Sort Algorithm

HEAPSORT (A)
1. BUILD_HEAP (A)
2. for i ← length (A) down to 2 do
exchange A[1] ↔ A[i]
heap-size [A] ← heap-size [A] - 1
Heapify (A, 1)

The HEAPSORT procedure takes time O(n lg n), since the call to
BUILD_HEAP takes time O(n) and each of the n -1 calls to Heapify takes
time O(lg n).

June 13, 2022 Sachin Singh DAA Unit I 69

 Sorting(CO1)
Example: A=[7, 4, 3, 1, 2]

June 13, 2022 Sachin Singh DAA Unit I 70

 Sorting(CO1)
Counting Sort

• Counting sort is a stable sorting technique, which is used

to sort objects according to the keys that are small numbers.

• It counts the number of keys whose key values are same.

• This sorting technique is effective when the difference between

different keys are not so big, otherwise, it can increase the space
complexity

• It works by counting the number of objects having distinct key values

(kind of hashing).

• Then there will be some arithmetic to calculate the position of each

object in the output sequence.
June 13, 2022 Sachin Singh DAA Unit I 71
 Sorting(CO1)

Counting Sort

Depends on a key assumption: numbers to be sorted are integers in

{0, 1, . . . , k}.

• Input: A[1 . . n], where A[ j ] Î {0, 1, . . . , k} for j = 1, 2, ..., n. Array A

and values n and k are given as parameters.

• Output: B[1 . . n], sorted. B is assumed to be already allocated and is

given as a parameter.

• Auxiliary storage: C[0 . . k]

June 13, 2022 Sachin Singh DAA Unit I 72

 Sorting(CO1)
COUNTING_SORT (A, B, k)
1. for i ← 0 to k do
2. c[i] ← 0
3. for j ← 1 to length[A] do
4. c[A[j]] ← c[A[j]] + 1
5. //c[i] now contains the number of elements equal to i
6. for i ← 1 to k do
7. c[i] ← c[i] + c[i-1]
8. // c[i] now contains the number of elements ≤ i
9. for j ← length[A] downto 1 do
10. B[c[A[j]]] ← A[j]
11. c[A[j]] ← c[A[j]] – 1

Analysis-The average time complexity for Bucket Sort is O(n + k).

The worst time complexity is O(n²). The
space complexity for Bucket Sort is O(n+k).
June 13, 2022 Sachin Singh DAA Unit I 73
 Sorting(CO1)
Counting Sort

June 13, 2022 Sachin Singh DAA Unit I 74

 Sorting(CO1)
Radix Sort

• The idea of Radix Sort is to do digit by digit sort starting from

least significant digit to most significant digit.
• Radix sort uses counting sort as a subroutine to sort.
• Radix sort is a non-comparative integer sorting algorithm that
sorts data with integer keys by grouping keys by the individual
digits which share the same significant position and value.

• To sort d digits:
RADIX_SORT (A, d)
for i ← 1 to d do
use a stable sort to sort A on digit i
// counting sort will do the job

June 13, 2022 Sachin Singh DAA Unit I 75

 Sorting(CO1)

Radix Sort:
Following example shows how Radix sort operates on four 3-digits
number.

Radix sort complexity is O(kn) for n keys which are integers of word
size k. For all there cases time i.e best , worst and average time
complexity is O(kn).

June 13, 2022 Sachin Singh DAA Unit I 76

 Sorting(CO1)
Bucket Sort
• Bucket sort, is a sorting algorithm that works by partitioning an array
into a number of buckets.

• Each bucket is then sorted individually, by using a different sorting

algorithm.

• Divide [0, 1) into n equal-sized buckets.

• Distribute the n input values into the buckets.

• Sort each bucket.

• Then go through buckets in order, listing elements in each one

June 13, 2022 Sachin Singh DAA Unit I 77

 Sorting(CO1)

BUCKET_SORT (A)

1. n ← length [A]
2. For i = 1 to n do
3. Insert A[i] into list B[nA[i]]
4. For i = 0 to n-1 do
5. Sort list B with Insertion sort
6. Concatenate the lists B[0], B[1], . . B[n-1] together in order.

• Time Complexity: O(n + k) for best case and average case and
O(n^2) for the worst case.
• Space Complexity: O(nk) for worst case

June 13, 2022 Sachin Singh DAA Unit I 78

 Sorting(CO1)
Bucket Sort

June 13, 2022 Sachin Singh DAA Unit I 79

 Daily Quiz

Q1) Process of inserting an element in stack is called

____________

Q2) Process of removing an element from stack is called

__________

Q3) Master’s theorem is used for?

Q4) How many cases are there under Master’s theorem?

Q5) In recursion, the condition for which the function will stop
calling itself is ____________

Q6) What is the average case running time of an insertion

sort algorithm?
June 13, 2022 Sachin Singh DAA Unit I 80
 Daily Quiz

Q7) What is the running time of an insertion sort algorithm if the input
is pre-sorted?

Q8) In C, what are the basic loops required to perform an insertion

sort?

Q9)Which of the following sorting algorithm is best suited if the

elements are already sorted?

Q10) What is recurrence for worst case of QuickSort and what is the
time complexity in Worst case?

June 13, 2022 Sachin Singh DAA Unit I 81

 Weekly Assignment

Q1 Solve the recurrence relation by iteration T (n) = T (n-1) + n 4 [CO1]

Q2 Rank the following by growth rate 2 lg n, (√2)lg n, log n!, log (logn),
log2n, (1/3) n, n1/lg n, (3/2) n [CO1]
Q3 Solve the recurrence: T (n) = 50 T (n/49) + log n!
[CO1]
Q4 Solve the following recurrence: T (n) = √n T (√n) + n
[CO1]
Q5 Use the master method to give tight asymptotic bounds for the
following recurrence. T (n) = 4T (n/2) + n.
[CO1]
Q6 Solve the recurrence T (n) = 2 T (√n) + 1 by making a change of
variables. Your solution should be asymptotically tight. Do not worry
about whether values are integral
[CO1]
Q7 Solve any two of the following
June 13, 2022 Sachin Singh
recurrences
DAA
by using most suitable
Unit I 82
method for solving recurrence relations. Your solution should be
 Weekly Assignment

Q8) How will you sort following array of 5 elements using heap sort
5, 9, 1, 17 and 6. [CO1]
Q9) llustrate the operation of INSERTION-SORT on the array
A = 31, 41, 59, 26, 41, 58 [CO1]
Q10) What do you mean by ‘Stable sorting algorithms’? Quick sort is
unstable where as merge is an stable sorting algorithm. Do you
agree with the above statement? Justify your answer. [CO1]
Q11) Analyze the running time of quick sort in the average case.
Q12) What is time complexity of counting sort? Sort 1, 9, 3, 3, 4, 5, 6,
7, 7, 8 by counting sort. [CO1]
Q13) Find out the worst case running time of merge sort. [CO1]

June 13, 2022 Sachin Singh DAA Unit I 83

 Faculty Video Links, You tube & NPTEL
Video Links and Online Courses Details
• You tube/other Video Links
• https://www.youtube.com/watch?v=yRM3sc57q0c&list=PLXFMml
k03Dt7Q0xr1PIAriY5623cKiH7V
• https://www.youtube.com/watch?v=A03oI0znAoc
• https://www.youtube.com/watch?v=Nd0XDY-jVHs
• https://www.youtube.com/watch?v=4V30R3I1vLI
• https://www.youtube.com/watch?v=IawM82BQ4II
• https://www.youtube.com/watch?v=1K9ebQJosvo
• https://www.youtube.com/watch?v=OynWkEj0S-s
• https://www.youtube.com/watch?v=CyknhZbfMqc

June 13, 2022 Sachin Singh DAA Unit I 84

 Faculty Video Links, You tube & NPTEL
Video Links and Online Courses Details

• You tube/other Video Links

• https://www.youtube.com/watch?v=BO145HIUHRg
• https://www.youtube.com/watch?v=mB5HXBb_HY8
• https://www.youtube.com/watch?v=6pV2IF0fgKY
• https://www.youtube.com/watch?v=7h1s2SojIRw
• https://www.youtube.com/watch?v=HqPJF2L5h9U
• https://www.youtube.com/watch?v=JMlYkE8hGJM
• https://www.youtube.com/watch?v=pEJiGC-ObQE

June 13, 2022 Sachin Singh DAA Unit I 85

 MCQ s
Q1. Two main measures for the efficiency of an algorithm are
a. Processor and memory b. Complexity and capacity
c. Time and space d. Data and space

Q2. The Worst case occur in linear search algorithm when

a. Item is somewhere in the middle of the array
b. Item is not in the array at all
c. Item is the last element in the array
d. Item is the last element in the array or is not there at all

Q3. The complexity of the average case of an algorithm is

a. Much more complicated to analyze than that of worst case
b. Much simpler to analyze than that of worst case
c. Sometimes more complicated and some other times simpler than
that of worst case
d. None or above
June 13, 2022 Sachin Singh DAA Unit I 86
 MCQ s

Q4. Which of the following sorting algorithms is the fastest?

(a) Merge sort (b) Quick sort (c) Insertion sort (d) Shell sort
Q5. What is the worst case time complexity of a quick sort algorithm?
(a) O(N) (b) O(N log N) (c) O(N2) (d) O(log N)
Q6. Quick sort is a __________
(a) greedy algorithm
(b) divide and conquer algorithm
(c) dynamic programming algorithm
(d) backtracking algorithm
Q7. What is the worst case time complexity of merge sort?
(a) O(n log n) (b) O(n2) (c) O(n2 log n) (d) O(n log
n2)

June 13, 2022 Sachin Singh DAA Unit I 87

 Glossary Question

Q.1 The complexity of searching an element from a set of n elements using Binary search
algorithm is_______
a. O(n log n)
b. O(log n)
c. O(n2) Incorrect
d. O(n)
Q.2______ is a condition that is always true at a particular point in an algorithm.
a. assertion
b. constant
c. exception
d. invariant Correct
Q.3 The running time of quick sort depends on the _________ .
a. Selection of pivot elements
b. Number of input
c. Number of passes
d. Arrangements of the elements

June 13, 2022 Sachin Singh DAA Unit I 88

 Glossary Question

Q.7 __________ is the worst case running time of shell sort, using Shell’s increments
a) O(N)
b) O(N log N)
c) O(log N)
d) O(N2)
Q.8 Heap sort is an implementation of ____________ using a descending priority
queue.
a) insertion sort
b) selection sort
c) bubble sort
d) merge sort
Q.9 The descending heap property is ___________
a) A[Parent(i)] = A[i]
b) A[Parent(i)] <= A[i]
c) A[Parent(i)] >= A[i]
d) A[Parent(i)] > 2 * A[i]

June 13, 2022 Sachin Singh DAA Unit I 89

 Glossary Question

Q.10 ______is wort case time complexity of Heap sort?

a) O(nlogn)
b) O(n2logn)
c) O(n2)
d) O(n3)

Q.11 In insertion sort, the average number of comparisons required to place the
7th element into its correct position is ______.
a) 9
b) 4
c) 7
d) 14
Q.12 _______ is the average case complexity of selection sort?
a) O(nlogn)
b) O(logn)
c) O(n)
d) O(n2)

June 13, 2022 Sachin Singh DAA Unit I 90

 Old Question Papers(2019-2020)

June 13, 2022 Sachin Singh DAA Unit I 91

 Old Question Papers(2019-2020)

June 13, 2022 Sachin Singh DAA Unit I 92

 Old Question Papers(2019-2020)

June 13, 2022 Sachin Singh DAA Unit I 93

 Old Question Papers(2019-2020)

June 13, 2022 Sachin Singh DAA Unit I 94

 Old Question Papers(2019-2020)

June 13, 2022 Sachin Singh DAA Unit I 95

 Old Question Papers(2019-2020)

June 13, 2022 Sachin Singh DAA Unit I 96

 Old Question Papers(2018-2019)

June 13, 2022 Sachin Singh DAA Unit I 97

 Old Question Papers(2018-2019)

June 13, 2022 Sachin Singh DAA Unit I 98

 Old Question Papers(2018-2019)

June 13, 2022 Sachin Singh DAA Unit I 99

 Old Question Papers(2018-2019)

June 13, 2022 Sachin Singh DAA Unit I 100

 Old Question Papers(2018-2019)

June 13, 2022 Sachin Singh DAA Unit I 101

 Old Question Papers(2017-2018)

June 13, 2022 Sachin Singh DAA Unit I 102

 Old Question Papers(2017-2018)

June 13, 2022 Sachin Singh DAA Unit I 103

 Old Question Papers(2017-2018)

June 13, 2022 Sachin Singh DAA Unit I 104

 Old Question Papers(2017-2018)

June 13, 2022 Sachin Singh DAA Unit I 105

 Old Question Papers(2017-2018)

June 13, 2022 Sachin Singh DAA Unit I 106

 Expected Questions for University Exam
Q1 Differentiate between asymptotic notation O and Ω. [CO1]
Q2 Use a recursion tree to give an asymptotically tight solution to the
recurrence T(n) = T(αn) + T((1 - α)n) + cn, where α is a constant in
the range 0 <α< 1 and c> 0 is also a constant. [CO1]
Q3 Find the time complexity of the recurrence relation
T(n)= n +T(n/10)+T(7n/5) [CO1]
Q4 Compare Time Complexity with Space Complexity. [CO1]
Q5 What are the characteristics of the algorithm? [CO1]
Q6 Show that the solution to T (n) = 2T (⌊n/2 ⌋ + 17) + n is O (n lg n).
[CO1]
Q7 Solve the recurrence: T (n) = 50 T (n/49) + log n!
[CO1]
Q8 Solve the recurrence using recursion tree method:
[CO1]
T (n) = T (n/2) + T (n/4) + T (n/8) + n
June 13, 2022 Sachin Singh DAA Unit I 107
 Expected Questions for University Exam

Q9. Write an algorithm to sort the given array of dement using Quick-
sort. Illustrate the operation of PARTITION procedure on the
array = < 2, 8, 7, 1, 3, 5, 6, 4 > . [CO1]
Q10. Apply BUCKET SORT algorithm on the following array 0 ∙ 78,
0 ∙ 17, 0 ∙ 39, 0 ∙ 26, 0 ∙ 72, 0 ∙ 94, 0 ∙ 21, 0 ∙ 21, 0 ∙ 12, 0 ∙ 23, 0 ∙ 68
[CO1]
Q11. Why Counting sort is called stable sort. [CO1]
Q12. Distinguish between Quick sort and Merge sort, and arrange the
following numbers in increasing order using merge sort
18, 29, 68, 32, 43, 37, 87, 24, 47, 50. [CO1]
Q13. What is divide and conquer strategy and explain the binary
search with suitable example. [CO1]

June 13, 2022 Sachin Singh DAA Unit I 108

 Recap Of Unit

This is an introductory chapter of Design & Analysis of Algorithm

covering the concept, importance and characteristics of algorithms. The
complexity and its calculation has been explained. Further, recursion
and different methodologies to solve them have also been provided.
In other part of unit different sorting algorithm – Insertion, Heap sort,
Quick sort, bucket sort, Radix Sort and Counting sort have been
discussed along with their complexities.
.

June 13, 2022 Sachin Singh DAA Unit I 109

 Thank You

June 13, 2022 Sachin Singh DAA Unit I 110