ACADEMIC YEAR: 2024-25

REGULATION: 2023
COMMON TO CSE , IT , CYBER SECURITY & AIDS
23CY4401 - DESIGN AND ANALYSIS OF ALGORITHMS
UNIT-I INTRODUCTION
MATERIAL
Algorithms: Definitions and notations: - Asymptotic notations and its properties – worst case, best case
and average case analysis; big oh, small oh, omega and theta notations; Recursive algorithms and non-
recursive, Mathematical analysis of non-recursive and recursive algorithms, solving recurrence
equations.
Analysis of Sorting and Searching: Heap sort, insertion sort; linear, binary and Interpolation Search,
Algorithm visualization Tool- Sorting.

1.Algorithms: Definitions and notations

Algorithm: The word algorithm came from the name of a Persian mathematician Abu Jafar

Mohammed Ibn Musa Al Khowarizmi (ninth century). An algorithm is simply s set of rules used

to perform some calculations either by hand or more usually on a machine (computer).

1.1 Definition: An algorithm is a finite set of instructions that accomplishes a particular task.

Another definition is a sequence of unambiguous instructions for solving a problem i.e, for

obtaining a required output for any legitimate (genuine) input in a finite amount of time.

1.2 In addition all algorithms must satisfy the following criteria (characteristics).

1. Input: zero or more quantities are externally supplied as input.

Consider Fibonacci numbers program, here aim of the problem is to display ten Fibonacci numbers ,
No input is required; in the problem itself this is clearly mentioned as ten Fibonacci values. So zero
items required for input.

Another problem is displaying given numbers of evens, so user should accept how many events

required. Based on the user input the number of evens is to be displayed. So, one data

item is required as input.

2. Output: At least one quantity is produced by given algorithm as output.

In the case of Fibonacci numbers program after executing the program, first ten

Fibonacci values displayed as output.

In second case, based on user input it should display given number of evens. An input of

negative number is wrong, should display proper error message as output. So this program

displays at least one output as error message, or number if outputs that show given number of
ACADEMIC YEAR: 2024-25
REGULATION: 2023
steps.

3. Definiteness: Each instruction is clear and unambiguous i.e. each step must be easy to

understand and convey only a single meaning.

4. Effectiveness: each instruction must be very basic, so that it can be carried out by a

person using only pencil and paper.

This step is common in both Fibonacci and primes. For example, if user enters a negative

numbers as input in evens, if you have a step like

Step: If N < 0 then

Go to ERROR

A wrong instruction given as go to ERROR, those kinds of instructions should not be there

in an algorithm.

5. Finiteness: If we can trace out the instructions of an algorithm then for all cases, the

algorithm terminate after a finite number of steps.

Either in the case of Fibonacci or even numbers problem should be solved in some number

of steps. For example, continuous display or Fibonacci series without termination leads to

abnormal termination.

1.3 Notations:

They can be expressed in various forms, including natural language, pseudocode, flowcharts, and
programming languages. The effectiveness of an algorithm is often evaluated based on its efficiency
and complexity, which are typically analyzed using asymptotic notations such as Big O, Omega, and
Theta.
ACADEMIC YEAR: 2024-25
REGULATION: 2023
1.4 Performance Analysis:

Performance analysis or analysis of algorithms refers to the task of determining the

efficiency of an algorithm i.,e how much computing time and storage an algorithm requires to

run (or execute). This analysis of algorithm helps in judging the value of one algorithm over

another.

To judge an algorithm, particularly two things are taken into consideration

1. Space complexity

2. Time complexity.

Space Complexity: The space complexity of an algorithm (program) is the amount of

memory it needs to run to completion. The space needed by an algorithm has the following

components.

1. Instruction Space.

2. Data Space.

3. Environment Stack Space.

Instruction Space: Instruction space is the space needed to store the compiled version of the

program instructions. The amount of instruction space that is needed depends on factors such as

i). The compiler used to compile the program into machine code.

ii). The compiler options in effect at the time of compilation.

iii). The target computer, i.,e computer on which the algorithm run.

Note that, one compiler may produce less code as compared to another compiler, when the same
program is compiled by these two.

Data Space: Data space is the space needed to store all constant and variable values. Data

space has two components.

i). Space needed by constants, for example 0, 1, 2.134.

ii). Space needed by dynamically allocated objects such as arrays, structures, classes.

Environmental Stack Space: Environmental stack space is used during execution of

functions. Each time function is involved the following data are saved as the environmental
ACADEMIC YEAR: 2024-25
REGULATION: 2023
stack.

i). The return address.

ii). Value of local variables.

iii). Value of formal parameters in the function being invoked.

Environmental stack space is mainly used in recursive functions. Thus, the space requirement

of any program p may therefore be written as

Space complexity S(P) = C + Sp (Instance characteristics).

This equation shows that the total space needed by a program is divided into two parts.

 Fixed space requirements(C) is independent of instance characteristics of the inputs and

outputs.

-Instruction space

-Space for simple variables, fixed-size structure variables, constants.

 A variable space requirements (SP(1)) dependent on instance characteristics 1.

- This part includes dynamically allocated space and the recursion stack space.

Example of instance character is:

Examples: 1

Algorithm NEC (float x, float y, float z)

Return (X + Y +Y * Z + (X + Y +Z)) /(X+ Y) + 4.0;

In the above algorithm, there are no instance characteristics and the space needed by X, Y, Z is

independent of instance characteristics, therefore we can write,

S(XYZ) =3+0=3

One space each for X, Y and Z

 Space complexity is O(1).

Time Complexity
ACADEMIC YEAR: 2024-25
REGULATION: 2023
The time complexity of an algorithm is the amount of compile time it needs to run to
completion. We can measure time complexity of an algorithm in two approaches

1. Priori analysis or compile time

2. Posteriori analysis or run (execution) time.

In priori analysis before the algorithm is executed we will analyze the behavior of the algorithm. A
priori analysis concentrates on determining the order if execution of statements.

In Posteriori analysis while the algorithm is executed we measure the execution time. Posteriori
analysis gives accurate values but it is very costly.

As we know that the compile time does not depend on the size of the input. Hence, we will
confine ourselves to consider only the run-time which depends on the size of the input and this run-
time is denoted by TP(n). Hence

Time complexity T(P) = C + TP(n).

The time (T(P)) taken by a program P is the sum of the compile time and execution time. The compile
time does not depend on the instance characteristics, so we concentrate on the runtime of a program.
This runtime is denoted by tp (instance characteristics).

The following equation determines the number of addition, subtraction, multiplication, division
compares, loads stores and so on, that would be made by the code for p.

tp(n) = CaADD(n)+ CsSUB(n)+ CmMUL(n)+ CdDIV(n)+……………..

where n denotes instance characteristics, and Ca, Cs, Cm, Cd and so on…..

As denote the time needed for an addition, subtraction, multiplication, division and so on, and ADD,
SUB, MUL, DIV and so on, are functions whose values are the number of additions, subtractions,
multiplications, divisions and so on. But this method is an impossible task to find out time complexity.

Another method is step count. By using step count, we can determine the number if steps needed by a
program to solve a particular problem.

2. Asymptotic notations and its properties

Accurate measurement of time complexity is possible with asymptotic notation.

Asymptotic complexity gives an idea of how rapidly the space requirement or time requirement
grow as problem size increase. When there is a computing device that can execute 1000 complex
operations per second. The size of the problem is that can be solved in a second or minute or an
hour by algorithms of different asymptotic complexity. In general asymptotic complexity is a
ACADEMIC YEAR: 2024-25
REGULATION: 2023
measure of algorithm not problem. Usually the complexity of an algorithm is as a function
relating the input length to the number of steps (time complexity) or storage location (space
complexity). For example, the running time is expressed as a function of the input size ‘n’ as
follows.

f(n)=n4+100n2+10n+50 (running time)

There are four important asymptotic notations.

1. Big oh notation (O)

2. Omega notation

3. Theta notation

Let f(n) and g(n) are two non-negative functions.

2.1Big oh notation
Big oh notation is denoted by ‘O’. it is used to describe the efficiency of an algorithm. It is

used to represent the upper bound of an algorithms running time. Using Big O notation, we can

give largest amount of time taken by the algorithm to complete.

Definition: Let f(n) and g(n) be the two non-negative functions. We say that f(n) is said to be

O(g(n)) if and only if there exists a positive constant ‘c’ and ‘n0‘ such that,

f(n) for all non-negative values of n, where n≥n0.

Here, g(n) is the upper bound for f(n). Ex:

Let f(n) = 2n4 + 5n2 + 2n +3

< 2n4 + 5n4 + 2n4 +3n4

< (2+5+2+3)n4

< 12n4.
. 4
. . f(n)=12n
This implies g(n)=n4, n >1
... c=12 and n0 =1
. 4
. . f(n)=O(n)
n

The above definition states that the function ‘f’ is almost ‘c’ times the function ‘g’ when ‘n’
ACADEMIC YEAR: 2024-25
REGULATION: 2023
is greater than or equal to n0.

This notion provides an upper bound for the function ‘f’ i.,e, the function g(n) is an
upper bound on the value of f(n) for all n, where n≥ n0.

2.2 Big omega notation

Big omega notation is denoted by ‘ ’. It is used to represent the lower bound of an

algorithms running time. Using big omega notation we can give shortest amount of time taken by
the algorithm to complete.

Definition: The function f(n)= (g(n)) (read as for of n is omega of g of n) if and only if there
exist positive constants ‘c’ and ‘n0’ such that,

f(n) ≥ c*g(n) for all n, n≥n0

Example:
. 4
. . g(n)=n ,
c=2 and n0 =1
.
4
.

f
(
n
)
=

(
n

)
Let f(n) = 2n4 + 5n2 + 2n +3
f(n)
> 2n4 (for example as

n , lower order oterms c*g(n)

are insignificant)

.
. . f(n) > 2n4, n >1

n0
ACADEMIC YEAR: 2024-25
REGULATION: 2023

2.3 Big Theta notation

The big theta notation is denoted by ‘ . It is in between the upper bound and lower
bound of an algorithms running time.

Definition: Let f(n) and g(n) be the two non-negetive functions. We say that f(n) is said to
be (g(n)) if and only if there exists a positive constants ‘c1’ and ‘c2’, such that,

c1 f(n) c2g((n) for all non-negative values n, where n ≥ n0.

The above definition states that the function f(n) lies between ‘c1’times the function g(n)
and ‘c2’, times the function g(n) where ‘c1’ and ‘c2’ are positive constants.

This notation provides both lower and upper bounds for the function f(n) i.,e, g(n) is both
lower and upper bounds on the value of f(n), for large n. in other words theta notation says that
f(n) is both O(g(n)) and (g(n)) for all n, where n≥n0.

This function f(n) = (g(n)) iff g(n) is both upper and lower bound an f(n
ACADEMIC YEAR: 2024-25
REGULATION: 2023
Example:
c2*g(n)

f(n) = 2n4 + 5n2 + 2n +3 f(n)Polynomial

− nΟ(1)
2n4 2n4 + 5n2 + 2n +3 12n4
2n4 f(n) 12n4 , n 1
Expone
c1*g(n)
ntial
g(n) = n4
− 2Ο(n)
... c1=2, c2=12 and n0=1
n0 n
... f(n)=(n4)

Little ‘oh’ notation

Little oh notation is denoted by “o”. the asymptotic upper bound

provided by O-notation may or may not be asymptotically tight. The
bound 2n2 =O(n2) is asymptotically tight, but the bound 2n=O(n2) is
not. We use o-notation to denote an upper bound that is not
asymptotically tight.

Definition: f(n)=o(g(n)), iff f(n)<c.g(n) for any positive

constants c>0, n0>0 and n> n0.

Common Asymptotic Notations

Following is a list of some common asymptotic notations –

Constant − Ο(1)

Logarithmic − Ο(log n)

Linear − Ο(n)

n log n − Ο(n log n)

Quadratic − Ο(n2)

Cubic − Ο(n3)
ACADEMIC YEAR: 2024-25
REGULATION: 2023
2.4 Properties of Asymptotic Notations
General Properties:
If f(n) is O(g(n)) then a*f(n) is also O(g(n)) ; where a is a constant.
Example: f(n) = 2n²+5 is O(n²)
then 7*f(n) = 7(2n²+5)
= 14n²+35 is also O(n²)
Similarly this property satisfies for both Θ and Ω notation.

If f(n) is Θ(g(n)) then a*f(n) is also Θ(g(n)) ; where a is a constant.

If f(n) is Ω (g(n)) then a*f(n) is also Ω (g(n)) ; where a is a constant.
1. Reflexive Properties :
If f(n) is given then f(n) is O(f(n)).
Example: f(n) = n² ; O(n²) i.e O(f(n))
Similarly this property satisfies for both Θ and Ω notation.
If f(n) is given then f(n) is Θ(f(n)).
If f(n) is given then f(n) is Ω (f(n)).
2. Transitive Properties :
If f(n) is O(g(n)) and g(n) is O(h(n)) then f(n) = O(h(n)) .
Example: if f(n) = n , g(n) = n² and h(n)=n³
n is O(n²) and n² is O(n³)
then n is O(n³)
Similarly this property satisfies for both Θ and Ω notation.
If f(n) is Θ(g(n)) and g(n) is Θ(h(n)) then f(n) = Θ(h(n)) .
If f(n) is Ω (g(n)) and g(n) is Ω (h(n)) then f(n) = Ω (h(n))
3. Symmetric Properties :
If f(n) is Θ(g(n)) then g(n) is Θ(f(n)) .
Example: f(n) = n² and g(n) = n²
then f(n) = Θ(n²) and g(n) = Θ(n²)
This property only satisfies for Θ notation.
4. Transpose Symmetric Properties :
If f(n) is O(g(n)) then g(n) is Ω (f(n)).
Example: f(n) = n , g(n) = n²
then n is O(n²) and n² is Ω (n)
This property only satisfies for O and Ω notations.
5. Some More Properties :
If f(n) = O(g(n)) and f(n) = Ω(g(n)) then f(n) = Θ(g(n))
If f(n) = O(g(n)) and d(n)=O(e(n))
then f(n) + d(n) = O( max( g(n), e(n) ))
Example: f(n) = n i.e O(n)
d(n) = n² i.e O(n²)
then f(n) + d(n) = n + n² i.e O(n²)
If f(n)=O(g(n)) and d(n)=O(e(n))
then f(n) * d(n) = O( g(n) * e(n) )
Example: f(n) = n i.e O(n)
ACADEMIC YEAR: 2024-25
REGULATION: 2023
d(n) = n² i.e O(n²)
then f(n) * d(n) = n * n² = n³ i.e O(n³)

3. worst case, best case and average case analysis;

In algorithm analysis, understanding the performance of an algorithm under various conditions
is essential. The three primary analyses—worst case, best case, and average case—help evaluate how
an algorithm behaves with different inputs. Here, we will explore these concepts using the linear
search algorithm as an example.

3.1 Linear Search Algorithm Overview

The linear search algorithm checks each element in a list sequentially until it finds the target element
or reaches the end of the list. The steps for performing a linear search are as follows:

1. Start from the first element of the array.

2. Compare the current element with the target value.

3. If they match, return the index of the current element.

4. If they do not match, move to the next element.

5. Repeat steps 2-4 until either the target is found or all elements have been checked.

3.1.1 Worst Case Analysis

Definition: The worst case scenario represents the maximum time required for an algorithm to
complete its task.Example for Linear Search:

 Scenario: The target element is not present in the array or is located at the last position.

Steps:

 The algorithm will check every element in the array.

 If there are nn elements, it will take nn comparisons.

 Time Complexity: Θ(n)Θ(n)

3.1.2. Best Case Analysis

Definition: The best case scenario describes the minimum time required for an algorithm to complete
its task.Example for Linear Search:

 Scenario: The target element is located at the first position of the array.

Steps:

 The algorithm finds the target on the first comparison.

 Time Complexity: Θ(1)Θ(1)

ACADEMIC YEAR: 2024-25
REGULATION: 2023
3.1.3 Average Case Analysis

Definition: The average case scenario evaluates the expected performance of an algorithm across all
possible inputs.Example for Linear Search:

 Assumption: All positions of elements are equally likely.

 Scenario: On average, the target element will be found halfway through the list.
Steps:
 If there are nn elements, on average, it will take (n+1)/2(n+1)/2 comparisons.
 Time Complexity: Θ(n)Θ(n)
Summary Table

Case Description Time Complexity

Worst Case Target not found or at last position Θ(n)Θ(n)

Best Case Target found at first position Θ(1)Θ(1)

Average Case Target found on average halfway through Θ(n)Θ(n)

Analyzing algorithms through worst case, best case, and average case scenarios provides
valuable insights into their efficiency and reliability. For linear search, while it is straightforward
with clear complexities, understanding these distinctions helps in selecting appropriate algorithms
based on expected performance in various conditions.

4.Mathematical analysis of non-recursive and recursive algorithms

4.1 Mathematical analysis of non-recursive algorithms
EXAMPLE 1 Consider the problem of finding the value of the largest element in a list of n numbers.
For simplicity, we assume that the list is implemented as an array. The following is pseudocode of a
standard algorithm for solving the problem.
ALGORITHM MaxElement(A[0..n − 1])
//Determines the value of the largest element in a given array
//Input: An array A[0..n − 1] of real numbers
//Output: The value of the largest element in A maxval ← A[0]
for i ← 1 to n − 1 do
if A[i] > maxval maxval ← A[i]
return maxval
ACADEMIC YEAR: 2024-25
REGULATION: 2023
The obvious measure of an input’s size here is the number of elements in the array, i.e., n. The
operations that are going to be executed most often are in the algorithm’s for loop. There are two
operations in the loop’s body: the comparison A[i] > maxval and the
assignment maxval ← A[i]. Which of these two operations should we consider basic? Since the
comparison is executed on each repetition of the loop and the assignment is not, we should consider
the comparison to be the algorithm’s basic operation. Note that the number of comparisons will be
the same for all arrays of size n; therefore, in terms of this metric, there is no need to distinguish
among the worst, average, and best cases here.
Let us denote C(n) the number of times this comparison is executed and try to find a formula
expressing it as a function of size n. The algorithm makes one comparison on each execution of the
loop, which is repeated for each value of the loop’s variable i within the bounds 1 and n − 1,
inclusive. Therefore, we get the following sum for C(n):

This is an easy sum to compute because it is nothing other than 1 repeated n − 1 times. Thus,

Here is a general plan to follow in analyzing nonrecursive algorithms.

General Plan for Analyzing the Time Efficiency of Nonrecursive Algorithms

 Decide on a parameter (or parameters) indicating an input’s size.
 Identify the algorithm’s basic operation. (As a rule, it is located in the inner-most loop.)
 Check whether the number of times the basic operation is executed depends only on the size
of an input. If it also depends on some additional property, the worst-case, average-case, and,
if necessary, best-case efficiencies have to be investigated separately.
 Set up a sum expressing the number of times the algorithm’s basic operation is executed.
Using standard formulas and rules of sum manipulation, either find a closed-form formula for
the count or, at the very least, establish its order of growth.
In particular, we use especially frequently two basic rules of sum manipulation
ACADEMIC YEAR: 2024-25
REGULATION: 2023
4.2 Matthematical Analysis of Recursive Algorithm:

Mathematical analysis of recursive algorithms involves several systematic steps

to evaluate their time efficiency and performance. This analysis is crucial for understanding how the
algorithm behaves as the input size changes, particularly for algorithms like computing the factorial
of a number. Below are the steps involved in this analysis, along with a specific example of finding
the factorial.
General Steps for Analyzing Recursive Algorithms
1. Decide on Input Size Parameters: Identify one or more parameters that indicate the size of
the input. For factorial calculations, this parameter is typically nn, the number for which the
factorial is computed.
2. Identify the Basic Operation: Determine what constitutes the basic operation in the
algorithm. In the case of factorial computation, the basic operation is multiplication.
3. Evaluate Variability of Basic Operation: Assess whether the number of times the basic
operation is executed varies with different inputs of the same size. If it does, analyze worst-
case, average-case, and best-case scenarios separately.
4. Set Up a Recurrence Relation: Establish a recurrence relation that describes how many
times the basic operation is executed based on smaller instances of the problem. For
factorials, this can be expressed as:
M(n)=M(n−1)+1for n>0M(n)=M(n−1)+1for n>0
with M(0)=0M(0)=0.
5. Solve the Recurrence Relation: Solve the recurrence relation to determine its growth order
or find an exact solution if possible. In this case, solving M(n)M(n) gives:
M(n)=nM(n)=n
indicating that n multiplications are required to compute n!
Example: Recursive Factorial Function
The recursive definition for calculating n!n! (factorial of nn) can be expressed as:
 Base Case: F(0)=1F(0)=1
 Recursive Case: F(n)=F(n−1)×n for n≥1
Algorithm Implementation
def factorial(n):
if n == 0:
return 1
else:
return factorial(n - 1) * n
ACADEMIC YEAR: 2024-25
REGULATION: 2023
Time Complexity Analysis
From our recurrence relation M(n)=M(n−1)+1M(n)=M(n−1)+1:
 The base case M(0)=0M(0)=0.
 By solving this recurrence using backward substitution, we find that:
 M(1)=M(0)+1=1M(1)=M(0)+1=1
 M(2)=M(1)+1=2M(2)=M(1)+1=2
 Continuing this pattern leads to M(n)=nM(n)=n.
Thus, the overall time complexity for computing n!n! using this recursive algorithm is O(n),
indicating linear growth relative to input size
.This structured approach to analyzing recursive algorithms not only aids in understanding their
efficiency but also helps in optimizing them when necessary.
5. Solving recurrence equations.
 A recurrence relation is a mathematical equation that describes a sequence of values,
where each term in the sequence is defined in terms of previous terms.

 In the context of Design and Analysis of Algorithms (DAA), recurrence relations are
often used to analyze the time complexity of algorithms by expressing the time
complexity of a problem of size n in terms of the time complexity of smaller instances of
the same problem.

The general form of a recurrence relation is often expressed as:

T(n)=f(n)+T(g(n))
where:

 T(n) is the time complexity of the problem of size.

 f(n) is the time complexity of the basic operations at the current level,
 g(n) represents the size of subproblems at the next level.
There are Four methods commonly used to solve recurrence relations:
Substitution Method
Recursion Tree
Iterations Method
Master Theorem

 Substitution Method
 Guess the form of the solution.
 Use mathematical induction to prove that the guess is correct.
ACADEMIC YEAR: 2024-25
REGULATION: 2023
For example consider the recurrence

T (n) = T +n

We have to show that it is asymptotically bound by O (log n).

Solution:
For T (n) = O (log n)
We have to show that for some constant c
T (n) ≤c logn.

Put this in given Recurrence Equation.

Now,

T (n) ≤c log +1

≤c log + 1 = c logn-clog2 2+1

≤c logn for c≥1
Thus T (n) =O logn.

Example2. Consider the Recurrence

T (n) = 2T + n n>1

Find an Asymptotic bound on T.

Solution:

We guess the solution is O (n (logn)).Thus for constant 'c'. For T (n) ≤c

n log ..Put this in given Recurrence Equation.

Now,
T (n) ≤2c log +n

≤cnlogn-cnlog2+n
=cn logn-n (clog2-1)

≤cn logn for (c≥1)

Thus T (n) = O (n logn)

The time Complexity of Substitution Method is O(nlogn), and the space complexity is
ACADEMIC YEAR: 2024-25
REGULATION: 2023
often the same unless additional space complexities (like recursion stack) need to be
considered.
 Recursion Tree Method:
 Represent the recurrence relation as a tree.
 Analyze the total cost by summing up the costs of all levels of the tree.
 In this method, we draw a recurrence tree and calculate the time taken by every level of
the tree.
 Finally, we sum the work done at all levels.
 To draw the recurrence tree, we start from the given recurrence and keep drawing till
we find a pattern among levels.
 The pattern is typically arithmetic or geometric series. For
example, consider the recurrence relation

T(n) = T(n/4) + T(n/2) + cn2

cn2
/ \
T(n/4) T(n/2)

If we further break down the expression T(n/4) and T(n/2),

we get the following recursion tree.
Solution:

cn2
/ \
c(n2)/16 c(n2)/4
/ \ / \
T(n/16) T(n/8) T(n/8) T(n/4)

Breaking down further gives us following

cn2
/ \
c(n2)/16 c(n2)/4
ACADEMIC YEAR: 2024-25
REGULATION: 2023
/ \ / \
c(n2)/256 c(n2)/64 c(n2)/64 c(n2)/16
/ \ / \ / \ / \

 To know the value of T(n), we need to calculate the sum of tree

 nodes level by level.
 If we sum the above tree level by level,
 we get the following series T(n) = c(n^2 + 5(n^2)/16 + 25(n^2)/256) +
….
 The above series is a geometrical progression with a ratio of 5/16.
 To get an upper bound, we can sum the infinite series.
 We get the sum as (n2)/(1 – 5/16) which is O(n2).

Example: QuickSort Algorithm

Let's apply the recursion tree method to analyze the time and space complexity of the
QuickSort algorithm.
Time Complexity:
The partitioning step in QuickSort takes O(n) time, and the algorithm splits the array into two
subarrays of approximately equal size.
The recursion tree for QuickSort has a height of logn (assuming balanced partitions at each
level)= Number of levels × Cost per level
T(n)=Number of levels×Cost per level
=logT(n)=logn×O n)
T(n)=O(nlogn)

Space Complexity:
ACADEMIC YEAR: 2024-25
REGULATION: 2023
The space complexity of QuickSort is influenced by the recursion stack. At each level, the
space used is proportional to the size of the subproblems at that level = Number of
levels×Space per level
S(n)=Number of levels×Space per level log
S(n)=logn×O(logn)

log S(n)=O(logn)
The dominant term in the space complexity is log n, indicating a logarithmic growth in space
usage.

So ,
Time Complexity is 0(n). Space
Complexity is log n

Iteration Methods:
 It means to expand the recurrence and express it as a summation of terms of n and
initial condition.
Example1:
Consider the Recurrence T (n) = 1 if n=1
= 2T (n-1) if n>1
Solution:
T (n) = 2T (n-1)
= 2[2T (n-2)] = 22T (n-2)
= 4[2T (n-3)] = 23T (n-3)
= 8[2T (n-4)] = 24T (n-4) ….(Eq.1)
Repeat the procedure for i times T (n)
= 2i T (n-i)
Put n-I =1 or I = n-1 in …(Eq.1)
T (n) = 2n-1 T (1)
= 2n-1 .1 {T (1) =1 ........ given}
= 2n-1

 Master Theorem:
 The Master Theorem is a tool used in the analysis of algorithms, particularly those
that follow a divide-and-conquer approach.
ACADEMIC YEAR: 2024-25
REGULATION: 2023
 It provides a straightforward way to determine the time complexity of recursive
algorithms by expressing their time recurrence relations in a specific form.
The general form of a recurrence relation that can be solved using the Master Theorem
is: T(n) = aT(n/b) + f(n)

Here:
o T(n) is the time complexity of the algorithm for a problem of size n,
o a is the number of subproblems at each recursion level,
o b is the factor by which the problem size is reduced in each recursive call,
o f(n) is the cost of dividing the problem and combining the results.
o The Master Theorem has three cases, each of which provides a solution for the time
complexity T(n)
 Internal Quality Assurance Cell
ACADEMIC YEAR: 2024-25 AC-01, W.e.f: 01.06.2023
Format Number: 2c Ver: 1.0
REGULATION: 2023

Case1:

Case2:

Case 3:

sufficiently large n, then the time complexity is:

The Master Theorem provides a convenient way to analyze the time complexity of
many divide-and-conquer algorithms without going through the process of constructing
a recursion tree or using the substitution method. However, it is applicable to a specific
form of recurrence relations, and not all recurrence relations fit the Master Theorem's
framework. In such cases, other methods like the recursion.

6. Analysis of Sorting and Searching

Sorting and searching are fundamental operations in computer science, essential for data
management and retrieval. Sorting algorithms arrange data in a specific order (ascending or
descending). They can be categorized into two main types: internal sorting (where data is
entirely in memory) and external sorting (where data is too large to fit into memory).
Searching algorithms retrieve specific elements from data structures. They can be broadly
classified into two categories: linear search (sequential) and binary search (efficient).
6.1 Heap sort
 Internal Quality Assurance Cell
ACADEMIC YEAR: 2024-25 AC-01, W.e.f: 01.06.2023
Format Number: 2c Ver: 1.0
REGULATION: 2023
Heap sort is a highly efficient sorting algorithm that utilizes the heap data structure, specifically a
binary heap, to sort elements. This method is particularly effective due to its in-place sorting
capability and relatively good time complexity.

6.1.1 Algorithm

Heap sort consists of two main phases: building a heap from the input data and then sorting the
heap.

1. Build a Max Heap: Convert the input array into a max heap. In a max heap, the parent
node is always greater than or equal to its child nodes.

2. Sorting:

 Swap the root of the max heap (the largest element) with the last element of the
heap.

 Reduce the size of the heap by one and call the max-heapify function on the root
to maintain the heap property.

 Repeat this process until all elements are sorted.

6.1.2 Pseudocode

HeapSort(A):

BUILD-MAX-HEAP(A)

for i = length(A) downto 2:

swap A[1] with A[i]

A.heap-size = A.heap-size - 1

MAX-HEAPIFY(A, 1)

Time Complexity Analysis

The time complexity of heap sort can be analyzed based on three scenarios: best case, average
case, and worst case.

 Best Case: O(nlog⁡n)O(nlogn)

 Even in the best scenario, building a max heap takes O(n)O(n), and each
extraction of the maximum element takes O(log⁡n)O(logn) for nn elements.

 Average Case: O(nlog⁡n)O(nlogn)

 Internal Quality Assurance Cell
ACADEMIC YEAR: 2024-25 AC-01, W.e.f: 01.06.2023
Format Number: 2c Ver: 1.0
REGULATION: 2023
 The average case also involves building the heap and repeatedly extracting

elements, resulting in O(nlog⁡n)O(nlogn).

 Worst Case: O(nlog⁡n)O(nlogn)

 Similar to average and best cases, the worst-case scenario does not
exceed O(nlog⁡n)O(nlogn) due to the nature of heap operations.

Space Complexity

Heap sort is an in-place sorting algorithm, which means it requires only a constant amount of
additional space. Therefore, its space complexity is:

 Space Complexity: O(1)O(1)

6.1.3 Characteristics of Heap Sort

 In-place: Requires only a small constant amount of additional storage space.

 Not stable: The relative order of equal elements may not be preserved.

 Comparison-based: Utilizes comparisons to determine the order of elements.

6.1.4 Applications

Heap sort is widely used in various applications where memory usage is critical, such as:

 Implementing priority queues.

 Sorting large datasets where memory overhead must be minimized.

 Situations requiring guaranteed performance bounds (i.e., consistently O(nlogn) time).

6.2 Insertion sort

Insertion sort is a simple sorting algorithm that builds a final sorted array (or list) one item
at a time. It is particularly effective for small datasets or when the data is already partially sorted.

6.2.1Algorithm
The insertion sort algorithm follows these steps:
1. Start from the second element (index 1) since the first element is trivially sorted.
2. Compare the current element with the elements in the sorted sub-list (elements to its left).
3. Shift all larger elements in the sorted sub-list to the right to make space for the current
element.
4. Insert the current element into its correct position in the sorted sub-list.
5. Repeat until all elements are processed.
 Internal Quality Assurance Cell
ACADEMIC YEAR: 2024-25 AC-01, W.e.f: 01.06.2023
Format Number: 2c Ver: 1.0
REGULATION: 2023
6.2.2 Pseudocode
InsertionSort(A):
for i from 1 to length(A) - 1:
key = A[i]
j=i-1
while j >= 0 and A[j] > key:
A[j + 1] = A[j]
j=j-1
A[j + 1] = key
6.2.3 Time Complexity Analysis
The time complexity of insertion sort varies based on the arrangement of elements in the input array:
 Best Case: O(n)O(n)
 Occurs when the array is already sorted. The inner loop only runs once for each
element, making the overall time linear.
 Average Case: O(n2)O(n2)
 On average, for each of the nn elements, about half of them will need to be compared
and shifted, leading to quadratic complexity.
 Worst Case: O(n2)O(n2)
 Happens when the array is sorted in reverse order. Every new element has to be
compared with all previously sorted elements, resulting in maximum shifts.
6.2.4 Space Complexity
 Space Complexity: O(1)O(1)
Insertion sort is an in-place sorting algorithm, meaning it requires a constant amount of additional
memory space regardless of input size.
6.2.5 Characteristics of Insertion Sort
 Stable: Maintains the relative order of equal elements.
 In-place: Requires minimal additional storage.
 Adaptive: Efficient for data that is already partially sorted.
6.2.6 Applications
Insertion sort is commonly used in various scenarios:
 Sorting small datasets where its simplicity and efficiency can be advantageous.
 As part of more complex algorithms like Timsort (used in Python's sort functions).
 Internal Quality Assurance Cell
ACADEMIC YEAR: 2024-25 AC-01, W.e.f: 01.06.2023
Format Number: 2c Ver: 1.0
REGULATION: 2023
 Situations where data is frequently added and requires maintaining a sorted order (e.g., online

sorting).
6.3 linear, binary and Interpolation Search,
6.3.1 Linear Search
//linearSearch Function

int linearSearch(int data[], int length, int val) {

for (int i = 0; i <= length; i++) { if (val == data[i]) {

return i; }//end if

}//end for

return -1; //Value was not in the list }//end linearSearch Function

Worst Case:

The worse case for Linear Search is achieved if the element to be found is not in the list
at all. This would entail the algorithm to traverse the entire list and return nothing. Thus
the worst case running time is: O(N).

Average Case:

The average case is in short revealed by insinuating that the average element would be
somewhere in the middle of the list or N/2. This does not change since we are dividing by
a constant factor here, so again the average case would be: O(N).

Best Case:

The best case can be a reached if the element to be found is the first one in the list. This
would not have to do any traversing spare the first one giving this a constant time
complexity or: O(1).

6.3.2 Binary Search

Algorithm: Binary-Search(numbers[], x, l, r)
if l = r then
return l
else
m := ⌊(l + r) / 2⌋
if x ≤ numbers[m] then
return Binary-Search(numbers[], x, l, m)
else
 Internal Quality Assurance Cell
ACADEMIC YEAR: 2024-25 AC-01, W.e.f: 01.06.2023
Format Number: 2c Ver: 1.0
REGULATION: 2023
return Binary-Search(numbers[], x, m+1, r)
Analysis
Linear search runs in O(n) time. Whereas binary search produces the result in O(log
n) time. Let T(n) be the number of comparisons in worst-case in an array of n elements.
Hence,

Using this recurrence relation T(n)=logn.

Therefore, binary search uses O(logn) time.

6.3.3 Interpolation Search

Interpolation Search is an advanced searching algorithm that enhances the efficiency of
searching in sorted arrays, particularly when the data is uniformly distributed. Unlike traditional
methods such as Binary Search, which consistently splits the search space in half, Interpolation
Search estimates the probable position of the target value based on its value relative to the values
at the extremes of the current search range.
How Interpolation Search Works
The algorithm operates under the assumption that the elements are uniformly distributed. It
calculates the estimated position of the target value using the following formula:

This
formula allows the algorithm to skip over large sections of the array, making it more efficient
than linear search, especially when dealing with large datasets. The steps involved in the
algorithm are as follows:
1. Initialization: Set low to 0 and high to n−1n−1 (the last index of the array).
2. Probing: Calculate the probable position using the interpolation formula.
3. Comparison: Check if the value at this position matches the target:
 If it does, return the index.
 If the target is greater, adjust low to pos + 1.
 If the target is smaller, adjust high to pos - 1.
4. Repeat until the target is found or low exceeds high.
 Internal Quality Assurance Cell
ACADEMIC YEAR: 2024-25 AC-01, W.e.f: 01.06.2023
Format Number: 2c Ver: 1.0
REGULATION: 2023
Complexity Analysis
 Average Case: The average time complexity of interpolation search is O(logl logN),
which is significantly better than linear search.
 Worst Case: In cases where the elements are not uniformly distributed, performance can
degrade to O(N), particularly if there are large gaps between value.
Advantages and Limitations
Advantages:
 Efficiency: Particularly effective for large datasets with uniformly distributed values.
 Reduced Comparisons: Can potentially reduce the number of comparisons needed
compared to binary search.
Limitations:
 Uniform Distribution Requirement: The algorithm performs poorly on non-uniformly
distributed datasets, where it may even be slower than linear search.
 Complexity in Implementation: The calculation of position adds complexity compared
to simpler algorithms like binary search

7 Algorithm visualization Tool- Sorting.

Algorithm visualization tools for sorting algorithms provide an interactive way to understand
how different sorting techniques operate on data. Here are some key points regarding these tools:
Purpose of Visualization Tools
 Enhance Understanding: Visualizing sorting algorithms helps users grasp the mechanics of
how algorithms like Bubble Sort, Quick Sort, and Merge Sort function.
 Engagement: Interactive tools engage users more effectively than static code or text
explanations.
Key Features
 Animation: Most visualization tools animate the sorting process, showing how elements are
compared and swapped in real-time.
 Speed Control: Users can often adjust the speed of the animations to observe each step clearly.
 Data Set Variability: Many tools allow users to modify the size and type of data sets (e.g.,
random, sorted, reverse sorted) to see how different algorithms perform under varying
conditions.
Popular Visualization Tools
 Internal Quality Assurance Cell
ACADEMIC YEAR: 2024-25 AC-01, W.e.f: 01.06.2023
Format Number: 2c Ver: 1.0
REGULATION: 2023
1. GeeksforGeeks Bubble Sort Visualization: This tool uses graphics to illustrate Bubble Sort,
where adjacent elements are swapped until the array is sorted. It employs color coding to
indicate current positions and swaps
2. Sorting Algorithms Visualizer (React): A modern web application that visualizes multiple
sorting algorithms with responsive design, allowing users to toggle between display formats
and adjust parameters.
3. Toptal Sorting Animations: This platform provides animations for various sorting algorithms
under different initial conditions, illustrating their performance characteristics.
4. Sort Visualizer: Offers visualizations for over 15 sorting algorithms, highlighting their
operational differences and complexities
Benefits of Using Visualization Tools
 Intuitive Learning: Visual representations make complex algorithms more accessible and
easier to comprehend.
 Comparison of Algorithms: Users can visually compare the efficiency and behavior of
different sorting methods in real-time.

 Error Diagnosis: Visual tools can help in debugging and understanding why certain algorithms
 perform poorly with specific data sets.
 Internal Quality Assurance Cell
ACADEMIC YEAR: 2024-25 AC-01, W.e.f: 01.06.2023
Format Number: 2c Ver: 1.0
REGULATION: 2023