DAA by ADI-1
DAA by ADI-1
of Algorithms
Algorithm
Characteristics
Problem Solving
Algorithm Design
Algorithm Analysis
Asymptotic Notations
3
INTRODUCTION
Algorithm:
A finite set of instruction that specifies a sequence of operation is to be carried out in
order to solve a specific problem or class of problems is called an Algorithm.
The word algorithm comes from the name of a Persian author, Abu Ja'far Mohammed
ibn Musa al Khowarizmi (c. 825 A.D.), who wrote a textbook on mathematics.
Algorithms are generally created independent of underlying languages.
The notion of the algorithm illustrates some important points:
The non-ambiguity requirement for each step of an algorithm cannot be compromised.
The range of inputs for which an algorithm works has to be specified carefully.
There may exist several algorithms for solving the same problem.
Algorithms for the same problem can be based on very different ideas and can solve
the problem with dramatically different speeds.
Prof. Dr. K. Adisesha
4
INTRODUCTION
Algorithm:
A finite set of instruction that specifies a sequence of operation is to be carried out in
order to solve a specific problem or class of problems is called an Algorithm.
Steps for writing an algorithm:
An Algorithm is a procedure. It has two part: head and body section.
The Head section consists of keyword Algorithm and Name of the algorithm with
parameter list. E.g. Algorithm name1(p1, p2,…,p3)
In the body of an algorithm various programming constructs like if, for, while and
some statements like assignments are used. The compound statements may be
enclosed with { and } brackets.
Comments are written using // at the beginning.
Prof. Dr. K. Adisesha
5
Algorithm
Euclid’s algorithm:
Euclid’s algorithm is based on applying repeatedly the equality gcd(m, n) = gcd(n, m
mod n), where m mod n is the remainder of the division of m by n, until m mod n is
equal to 0.
Euclid’s algorithm for computing gcd(m, n) in simple steps
Step 1: If n = 0, return the value of m as the answer ALGORITHM Euclid_gcd(m, n)
and stop; otherwise, proceed to Step 2. //Computes gcd(m, n) by Euclid’s algorithm
//Input: Two non-zero integers m and n
Step 2: Divide m by n and assign the value of the //Output: Greatest common divisor of m and n
remainder to r. while n ≠ 0 do
Step 3: Assign the value of n to m and the value of r r ←m mod n
to n. Go to Step 1. m←n
n←r
Prof. Dr. K. Adisesha return m
6
Algorithm
Euclid’s algorithm:
Euclid’s algorithm is based on applying repeatedly the equality gcd(m, n) = gcd(n, m
mod n), where m mod n is the remainder of the division of m by n, until m mod n is
equal to 0.
Euclid’s algorithm is based on repeated application of equality
gcd(m,n) = gcd(n, m mod n)
Example:
Euclid’s algorithm for computing gcd(60, 24) in simple steps
gcd(60,24) = gcd(24,12) = gcd(12,0) = 12
Characteristics of an Algorithm:
An algorithm should have the following characteristics −.
Unambiguous − Algorithm should be clear and unambiguous. Each of its steps (or
phases), and their inputs/outputs should be clear and must lead to only one meaning.
Input: Zero / more quantities are externally supplied.
Output: At least one quantity is produced.
Definiteness: Each instruction is clear and unambiguous.
Finiteness: If the instructions of an algorithm is traced then for all cases the algorithm
must terminates after a finite number of steps.
Efficiency: Every instruction must be very basic and runs in short time.
Problem Types :
The most important problem types are:
Sorting
Searching
String processing
Graph problems
Combinatorial problems
Geometric problems
Numerical problems
Design Strategies:
There are various types of strategies in order to design algorithms for various
problems. Some of them are listed as follows:
Divide & Conquer Approach
Greedy Approach
Dynamic Programming Approach
Randomization Approach
Approximation Approach
Recursive Approach
Branch and Bound Approach
Design Strategies:
There are various types of strategies in order to design algorithms for various
problems. Some of them are listed as follows:
Divide and Conquer Approach: It is a top-down approach. The algorithms which
follow the divide & conquer techniques involve three steps:
Divide the original problem into a set of subproblems.
Solve every subproblem individually, recursively.
Combine the solution of the subproblems (top level) into a solution of the whole original
problem.
Greedy Technique: Greedy method is used to solve the optimization problem in which
we are given a set of input values, which are required either to be maximized or
minimized (known as objective), i.e. some constraints or conditions.
Prof. Dr. K. Adisesha
14
Algorithm
Design Strategies:
There are various types of strategies in order to design algorithms for various
problems. Some of them are listed as follows:
Dynamic Programming: Dynamic Programming is a bottom-up approach we solve all
possible small problems and then combine them to obtain solutions for bigger problems.
Branch and Bound: In Branch & Bound algorithm a given subproblem, which cannot be
bounded, has to be divided into at least two new restricted subproblems.
Randomized Algorithms: A randomized algorithm is defined as an algorithm that is
allowed to access a source of independent, unbiased random bits, and it is then allowed to
use these random bits to influence its computation.
Backtracking Algorithm: Backtracking Algorithm tries each possibility until they find the
right one. It is a depth-first search of the set of possible solution.
Prof. Dr. K. Adisesha
15
Algorithm
Algorithm Analysis:
Efficiency of an algorithm can be analyzed at two different stages, before
implementation and after implementation. They are the following .
A Priori Analysis − This is a theoretical analysis of an algorithm. Efficiency of an
algorithm is measured by assuming factors, like processor speed, are constant and have
no effect on the implementation.
Algorithm Complexity:
Step Count Method− The step Count method is also called as Frequency Count method.
we count the number of times each instruction is executed.
Calculate
N
i
i 1
3 1
2
1
2N+2
3 4N
Lines 1 and 4 count for one unit each 4 1
Line 3: executed N times, each time four units
Line 2: (1 for initialization, N+1 for all the tests, N for all the increments) total 2N + 2
total cost: 6N + 4 O(N)
19
Algorithm
Analysis of Algorithms:
To analyse the feasibility of an algorithm designed, we calculate the complexity of it.
This is represented in three notations, called asymptotic notations. Asymptotic
notations are as follows −
Worst-Case Scenario − Big Oh and Little Oh Notations
Best-Case Scenario − Big Omega and Little Omega Notations
Average-Case Scenario − Theta Notation
θ(f(n)) = { g(n) if and only if g(n) = Ο(f(n)) and g(n) = Ω(f(n)) for all n > n0. }
Prof. Dr. K. Adisesha
25
Algorithm
Recurrence:
Recursion is a fundamental concept in computer science and mathematics that
allows functions to call themselves, enabling the solution of complex problems
through iterative steps.
There are four methods for solving Recurrence:
Substitution Method
Iteration Method
Recursion Tree Method
Master Method
For Example, the Worst Case Running Time T(n) of the
MERGE SORT Procedures is described by the recurrence.
Prof. Dr. K. Adisesha
29
Recurrence Relation
Iteration Methods:
It means to expand the recurrence and express it as a summation of terms of n and
initial condition.
Example: Consider the Recurrence:
T (n) = 1 if n=1
= 2T (n-1) if n>1 Repeat the procedure for i times
Solution: T (n) = 2i T (n-i)
T (n) = 2T (n-1) Put n-i=1 or i= n-1 in (Eq.1)
= 2[2T (n-2)] = 22T (n-2) T (n) = 2n-1 T (1)
= 4[2T (n-3)] = 23T (n-3) = 2n-1 .1 {T (1) =1 .....given}
= 8[2T (n-4)] = 24T (n-4) (Eq.1) = 2n-1
Prof. Dr. K. Adisesha
30
Recurrence Relation
Recursion Tree:
A recursion tree is a graphical representation that illustrates the execution flow of a
recursive function. It provides a visual breakdown of recursive calls.
Each node in a recursion tree represents a particular recursive call. The initial call is
depicted at the top, with subsequent calls branching out beneath it.
// find factorial of a number
/* How function calls are made,
factorial(n) { Factorial(5) [ 120 ]
if n is less than 2: // Factorial of 0, 1 is 1 5 * Factorial(4) ==> 120
return n 4. * Factorial(3) ==> 24
else // Recursive step 3 * Factorial(2) ==> 6
return n * factorial(n-1); // Factorial of 5 => 5 * Factorial(4)... 2 * Factorial(1) ==> 2
} 1
*/
Prof. Dr. K. Adisesha
31
Brute Force Approach
Linear search:
Linear Searching: Also called Sequential Searching.
It is used to find out the location of the data item if it exists in the given collection of
data items.
Example Searching element 33 from the array of elements:
Sorting in Arrays:
A Sorting Algorithm is used to rearrange a given array or list elements according to a
comparison operator on the elements:
The comparison operator is used to decide the new order of element in the respective
data structure.
Types of Sorting Algorithms are:
Bubble Sort Heap Sort
Selection Sort Radix Sort
Insertion Sort Bucket Sort
Merge Sort Shell Sort
Quick Sort
Prof. Dr. K. Adisesha
37
Brute Force Approach
Selection Sort:
Selection Sort:
Selection Sort:
Complexity Analysis of Selection Sort
Input: Given n input elements.
Output: Number of steps incurred to sort a list.
Logic: If we are given n elements, then in the
First pass, it will do n-1 comparisons;
Second pass, it will do n-2; in the third pass,
it will do n-3 and so on.
Therefore, the selection sort algorithm encompasses a time complexity of O(n2) and a
space complexity of O(1) because it necessitates some extra memory space for temp
variable for swapping.
Prof. Dr. K. Adisesha
41
Exhaustive search Approach
Exhaustive search:
Exhaustive search, also known as brute-force search or generate and test, is a problem-
solving technique that involves systematically checking all possible candidates to see if
they meet the problem's requirements.
Exhaustive Search:
Traveling Salesman Problem
Knapsack Problem
Assignment Problem
Graph Traversals
Knapsack Problem:
In the knapsack problem, you need to pack a set of items, with given values and sizes
(such as weights or volumes), into a container with a maximum capacity.
If the total size of the items exceeds the capacity, you can't pack them all. In that case,
the problem is to choose a subset of the items of maximum total value that will fit in the
container.
The data includes the following:
Weights: A vector containing the weights of the items.
Values: A vector containing the values of the items.
Capacities: A vector with just one entry, the capacity of the knapsack.
Knapsack Problem:
The Knapsack problem is an example of the combinational optimization problem. This
problem is also commonly known as the “Rucksack Problem“.
The knapsack problem can be classified into the following types:
Fractional Knapsack Problem
0/1 Knapsack Problem
Bounded Knapsack Problem
Unbounded Knapsack Problem
Example:
Wi={4, 3, 2, 1}
Vi={5, 4, 3, 2 } Capacity=6
Prof. Dr. K. Adisesha
53
Exhaustive search Approach
BFS algorithm:
Breadth-first search is a graph traversal algorithm that starts traversing the graph
from the root node and explores all the neighboring nodes.
The applications of breadth-first-algorithm are given as follows -
BFS can be used to find the neighboring locations from a given source location.
In a peer-to-peer network, BFS algorithm can be used as a traversal method to find all the
neighboring nodes. Most torrent clients, such as BitTorrent, uTorrent, etc. employ this
process to find "seeds" and "peers" in the network.
BFS can be used in web crawlers to create web page indexes. It is one of the main
algorithms that can be used to index web pages.
BFS is used to determine the shortest path and minimum spanning tree.
BFS is also used in Cheney's technique to duplicate the garbage collection.
Prof. Dr. K. Adisesha
56
Exhaustive search Approach
BFS algorithm:
Breadth-first search is a graph traversal algorithm that starts traversing the graph
from the root node and explores all the neighboring nodes.
The steps involved in the BFS algorithm to explore a graph are given as follows -
Step 1: SET STATUS = 1 (ready state) for each node in G
Step 2: Enqueue the starting node A and set its STATUS = 2 (waiting state)
Step 3: Repeat Steps 4 and 5 until QUEUE is empty
Step 4: Dequeue a node N. Process it and set its STATUS = 3 (processed state).
Step 5: Enqueue all the neighbours of N that are in the ready state (whose
STATUS = 1) and set their STATUS = 2
(waiting state)
[END OF LOOP]
Step 6: EXIT.
Prof. Dr. K. Adisesha
57
Exhaustive search Approach
Input: Profits[] = {60, 100, 120, 50} Input: Profits[] = {60, 100, 120, 50}
Weights[] = {10, 20, 30, 40}, W = 40 Weights[] = {10, 20, 30, 40}, W = 50
Output: Output:
10: 60, 20: 100, 10: 60, 20: 100,
10: 60, 30: 120, 10: 60, 30: 120,
Maximum Profit = 180 20: 100, 30: 120,
Explanation: Maximum profit from all the possible Maximum Profit = 220
solutions is 180 Explanation: Maximum profit from all the possible
solutions is 220
Prof. Dr. K. Adisesha
61
Decrease and Conquer
Binary Searching:
The binary search algorithm can be used with
only sorted list of elements.
Binary Search first divides a large array into
two smaller sub-arrays and then recursively
operate the sub-arrays.
Binary Search basically reduces the search
space to half at each step
Binary Searching:
Time to compare the search element with mid element, with half of the selected half part of array
Insertion Sorting:
Insertion sort is a simple sorting algorithm that builds the final sorted array (or list)
one item at a time.
This is an in-place comparison-based sorting algorithm. Here, a sub-list is maintained
which is always sorted.
Insertion Sorting:
This is an in-place comparison-based sorting algorithm. Here, a sub-list is maintained
which is always sorted.
This is an in-place comparison-based sorting algorithm. Here, a sub-list is maintained
which is always sorted.
Insertion Sorting:
ALGORITHM: Insertion Sort (A, N) A is an array with N unsorted elements.
Step 1: for I=1 to N-1
Step 2: J = I
While(J >= 1)
if ( A[J] < A[J-1] ) then
Temp = A[J];
A[J] = A[J-1];
A[J-1] = Temp;
[End if]
J = J-1
[End of While loop]
[End of For loop]
Prof. Dr. K. Adisesha Step 3: Exit
70
Divide and Conquer
Greedy Algorithm:
A greedy algorithm is an approach for solving a problem by selecting the best option
available at the moment. It works in a top-down approach.
This technique is basically used to determine the feasible solution that may or may not
be optimal.
The following are the characteristics of a greedy method:
To construct the solution in an optimal way, this algorithm creates two sets where one
set contains all the chosen items, and another set contains the rejected items.
A Greedy algorithm makes good local choices in the hope that the solution should be
either feasible or optimal.
Greedy Algorithm:
A greedy algorithm is an approach for solving a problem by selecting the best option
available at the moment. It works in a top-down approach.
The components that can be used in the greedy algorithm are:
Candidate set: A solution that is created from the set is known as a candidate set.
Selection function: This function is used to choose the candidate or subset which can be
added in the solution.
Feasibility function: A function that is used to determine whether the candidate or subset can
be used to contribute to the solution or not.
Objective function: A function is used to assign the value to the solution or the partial
solution.
Solution function: This function is used to intimate whether the complete function has been
reached or not.
Prof. Dr. K. Adisesha
84
Greedy Technique
Greedy Algorithm:
A greedy algorithm is an approach for solving a problem by selecting the best option
available at the moment. It works in a top-down approach.
The components that can be used in the greedy algorithm are:
Candidate set: A solution that is created from the set is known as a candidate set.
Selection function: This function is used to choose the candidate or subset which can be
added in the solution.
Feasibility function: A function that is used to determine whether the candidate or subset can
be used to contribute to the solution or not.
Objective function: A function is used to assign the value to the solution or the partial
solution.
Solution function: This function is used to intimate whether the complete function has been
reached or not.
Prof. Dr. K. Adisesha
85
Greedy Technique
Greedy Algorithm:
A greedy algorithm is an approach for solving a problem by selecting the best option
available at the moment. It works in a top-down approach.
Applications of Greedy Algorithm
It is used in finding the shortest path.
It is used to find the minimum spanning tree using the prim's algorithm or the
Kruskal's algorithm.
It is used in a job sequencing with a deadline.
This algorithm is also used to solve the fractional knapsack problem.
Greedy Algorithm:
A greedy algorithm is an approach for solving a problem by selecting the best option
available at the moment. It works in a top-down approach.
Types of Greedy Algorithms
Knapsack Problem.
Minimum Spanning Tree.
Single-Source Shortest Path Problem.
Job Scheduling Problem.
Prim's Minimal Spanning Tree Algorithm.
Kruskal's Minimal Spanning Tree Algorithm.
Dijkstra's Minimal Spanning Tree Algorithm.
Prof. Dr. K. Adisesha
87
Greedy Technique
Kruskal's Algorithm:
Kruskal's algorithm is a minimum spanning tree algorithm that takes a graph as input
and finds the subset of the edges of that graph which.
form a tree that includes every vertex
has the minimum sum of weights among all the trees that can be formed from the
graph
The steps for implementing Kruskal's algorithm are as follows:
Sort all the edges from low weight to high
Take the edge with the lowest weight and add it to the spanning tree. If adding the
edge created a cycle, then reject this edge.
Keep adding edges until we reach all vertices.
Prof. Dr. K. Adisesha
90
Greedy Technique
Kruskal's Algorithm:
Kruskal's algorithm is a subgraph that connects all the vertices present in the main graph
with the least possible edges and minimum cost.
Examples: Construct a minimum spanning tree using kruskal’s algorithm for the graph given
Solution:
As the first step, sort all the edges in the given graph in an
ascending order and store the values in an array.
Edge BD AB CF FE BC GF AG CD DE CG
Cost 5 6 9 10 11 12 15 17 22 25
Minimum cost = 5 + 6 + 9 + 10 + 11 + 12 = 53
Visited array, v = {B, D, A, C, F, E, G}
Prof. Dr. K. Adisesha
91
Greedy Technique
Prim's algorithm:
Prim's algorithm to find minimum cost spanning tree uses the greedy approach.
Algorithm
Declare an array visited[] to store the visited vertices and, add the arbitrary root, say S, to the visited
array.
Check whether the adjacent vertices of the last visited vertex are present in the visited[] array or not.
If the vertices are not in the visited[] array, compare the cost of edges and add the least cost edge to the
output spanning tree.
The adjacent unvisited vertex with the least cost edge is added into the visited[] array and the least cost
edge is added to the minimum spanning tree output.
Steps 2 and 4 are repeated for all the unvisited vertices in the graph to obtain the full minimum spanning
tree output for the given graph.
Calculate the cost of the minimum spanning tree obtained.
Prof. Dr. K. Adisesha
92
Greedy Technique
Prim's algorithm:
Prim's algorithm to find minimum cost spanning tree (as Kruskal's algorithm) uses the
greedy approach. Prim's algorithm shares a similarity with the shortest path first
algorithms.
The algorithm, similar to any shortest path algorithm, begins from a vertex that is set as a root
and walks through all the vertices in the graph by determining the least cost adjacent edges.
Step 1 - Remove all loops and parallel edges
Step 2 - Choose any arbitrary node as root node
Dijkstra’s Algorithm:
Dijkstra's shortest path algorithm was invented in 1956 by the Dutch computer scientist
Edsger W. Dijkstra during a twenty minutes coffee break, while out shopping with his
fiancée in Amsterdam.
Dijkstra's algorithm is used for solving single-source
shortest path problems for directed or undirected paths.
Dijkstra's algorithm finds the shortest path from one vertex
to all other vertices.
It does so by repeatedly selecting the nearest unvisited
vertex and calculating the distance to all the unvisited
neighboring vertices.
Prof. Dr. K. Adisesha
94
Greedy Technique
Dijkstra’s Algorithm:
Dijkstra's algorithm is used for solving single-source shortest path problems for directed
or undirected paths.
1. Set initial distances for all vertices: 0 to source vertex, and infinity for all the other.
2. Choose the unvisited vertex with the shortest distance from the start to be current vertex.
3. For each of the current vertex's unvisited neighbor vertices, calculate the distance from
the source and update the distance if the new, calculated, distance is lower.
4. We are now done with the current vertex, so we mark it as visited. A visited vertex is not
checked again.
5. Go back to step 2 to choose a new current vertex, and keep repeating these steps until all
vertices are visited.
6.Prof.InDr.the end we are left with the shortest path from the source vertex to every other vertex.
K. Adisesha
95
Greedy Technique
Dijkstra’s Algorithm:
Given a weighted graph and a source vertex in the graph, find the shortest paths from the
source to all the other vertices in the given graph.
Input: src = 0, the graph.
Output: 0 4 12 19 21 11 9 8 14
Explanation: The distance from 0 to 1 = 4.
The minimum distance from 0 to 2 = 12. 0->1->2
The minimum distance from 0 to 3 = 19. 0->1->2->3
The minimum distance from 0 to 4 = 21. 0->7->6->5->4
The minimum distance from 0 to 5 = 11. 0->7->6->5
The minimum distance from 0 to 6 = 9. 0->7->6
The minimum distance from 0 to 7 = 8. 0->7
The minimum distance from 0 to 8 = 14. 0->1->2->8
Prof. Dr. K. Adisesha
96
Greedy Technique
P Class Problem:
P-type problems are a class of problems in computational complexity theory that can be
solved by a deterministic Turing machine within a polynomial amount of time.
This is also known as polynomial-time deterministic algorithms.
P: the class of problems that have polynomial-time deterministic algorithms.
A deterministic algorithm is (essentially) one that always computes the correct
answer.
This problem is easy to understand and tractable.
Sample Problems in P
Searching
Element uniqueness
Graph connectivity
Prof. Dr. K. Adisesha
10
0
Greedy Technique
NP Class Problem:
A problem is said to be Nondeterministic polynomial time that can be solvable in
polynomial time by a nondeterministic Turing machine.
NP stands for “Nondeterministic Polynomial-time”
The solutions to the NP class problem are hard to find since they are being solved by a
non-deterministic machine.
Some of the examples of NP problems are given as follows:
Traveling Salesman
N-Queens
Classroom Scheduling
Packing
Scheduling
Prof. Dr. K. Adisesha
10
1
Greedy Technique
NP-Complete Problems:
NP-Complete (or NPC) problems are a set of problems that are well connected.
A problem x that is in NP, if any one finds an polynomial time algorithm even for one
problem, it implies that polynomial time algorithm for all NP-Complete problems.
NP-complete problems are a subset of the larger class of NP (nondeterministic
polynomial time) problems.
NP problems are a class of computational problems that can be solved in polynomial
time by a non-deterministic machine and can be verified in polynomial time by a
deterministic Machine.
N Queen Problem:
The N Queen is the problem of placing N chess queens on an N×N chessboard so that no
two queens attack each other.
Given an integer n, return all distinct solutions to the n-queens puzzle. You may return
the answer in any order.
Input: n = 4
Output: [[".Q..","...Q","Q...","..Q."],
["..Q.","Q...","...Q",".Q.."]]
Thank you:
s t
b e
th e
A l l