Greedy Algorithms

Greedy Algorithms
Greedy algorithms are a class of algorithms that make locally

optimal choices at each step with the hope of finding a global
optimum solution. In these algorithms, decisions are made based on the
information available at the current moment without considering the
consequences of these decisions in the future. The key idea is to select the
best possible choice at each step, leading to a solution that may not
always be the most optimal but is often good enough for many problems.
In this article, we will understand greedy algorithms with examples. We
will also look at problems and their solutions using the greedy approach.
What is Greedy Algorithm?

A greedy algorithm is a type of optimization algorithm that makes locally
optimal choices at each step to find a globally optimal solution. It operates
on the principle of “taking the best option now” without considering the
long-term consequences.
Steps for Creating a Greedy Algorithm
Elements of greedy strategy:-
The steps to define a greedy algorithm are:
1. Define the problem: Clearly state the problem to be solved and
the objective to be optimized.
2. Identify the greedy choice: Determine the locally optimal choice
at each step based on the current state.
3. Make the greedy choice: Select the greedy choice and update
the current state.
4. Repeat: Continue making greedy choices until a solution is
reached.
Following the given steps, one can learn how to use greedy algorithms to
find optimal solutions.
Greedy Algorithm Examples
Examples of greedy algorithms are the best way to understand the
algorithm. Some greedy algorithm real-life examples are:
• Fractional Knapsack: Optimizes the value of items that can be
fractionally included in a knapsack with limited capacity.
• Dijkstra’s algorithm: Finds the shortest path from a source
vertex to all other vertices in a weighted graph.
• Kruskal’s algorithm: Finds the minimum spanning tree of a
weighted graph.
• Huffman coding: Compresses data by assigning shorter codes
to more frequent symbols.
Applications of Greedy Algorithm
There are many applications of the greedy method in DAA. Some
important greedy algorithm applications are:
• Assigning tasks to resources to minimize waiting time or
maximize efficiency.
• Selecting the most valuable items to fit into a knapsack with
limited capacity.
• Dividing an image into regions with similar characteristics.
• Reducing the size of data by removing redundant information.
Disadvantages/Limitations of Using a Greedy Algorithm
Below are some disadvantages of the Greedy Algorithm:
• Greedy algorithms may not always find the best possible
solution.
• The order in which the elements are considered can significantly
impact the outcome.
• Greedy algorithms focus on local optimizations and may miss
better solutions that require considering a broader context.
• Greedy algorithms are not applicable to problems where the
greedy choice does not lead to an optimal solution.
Local optima are points in a problem space where a function reaches its
highest (or lowest) value within a small neighborhood. For example, in
a terrain with hills and valleys, a local peak represents a local optimum
because it's the highest point nearby.
Global optima, on the other hand, are points where the function
reaches its highest (or lowest) value across the entire problem space. In
the same terrain example, the highest peak represents the global
optimum because it's the highest point in the entire landscape.
In essence, local optima are "hills" in the solution space, while the
global optimum is the tallest "mountain" across the entire landscape.
The Matroid Greedy Algorithm is a method for finding an optimal

solution to certain combinatorial optimization problems. Matroids are
mathematical structures that generalize the notion of linear
independence in vector spaces. The Matroid Greedy Algorithm exploits
the properties of matroids to efficiently find solutions.
Here's a high-level overview of the Matroid Greedy Algorithm:
1. Definition of a Matroid: A matroid is defined by a finite ground

set and a collection of subsets of this ground set, called
independent sets, which satisfy the following properties:
• Non-emptiness: The empty set is an independent set.
• Heredity: If a set is independent, then any subset of it is also
independent.
• Exchange property: If set A is independent and has fewer
elements than set B, then there exists an element in B that
can be added to A without violating independence.
2. Greedy Selection: Given a matroid, the Matroid Greedy Algorithm
iteratively selects elements to form an independent set by
greedily choosing elements that satisfy the exchange property. At
each step, it selects the element that maximizes the objective
function while maintaining independence.
3. Termination: The algorithm terminates when it's no longer
possible to add any more elements to the current set without
violating independence.
4. Optimality: The solution obtained using the Matroid Greedy
Algorithm is guaranteed to be optimal for problems where the
objective function satisfies the following properties:
• Monotonicity: Adding elements to the set never decreases
the objective function value.
• Submodularity: The marginal increase in the objective
function value decreases as the size of the set increases.
5. Applications: Matroid Greedy Algorithm is applied to various
optimization problems, such as the Maximum Weighted
Independent Set, Maximum Weighted Spanning Tree, and
Maximum Weighted Matching.
Overall, the Matroid Greedy Algorithm offers an efficient and elegant

approach to solving combinatorial optimization problems by leveraging
the structure and properties of matroids.
Activity Selection Problem

Greedy is an algorithmic paradigm that builds up a solution piece by
piece, always choosing the next piece that offers the most obvious and
immediate benefit. Greedy algorithms are used for optimization
problems.
An optimization problem can be solved using Greedy if the problem has
the following property:
• At every step, we can make a choice that looks best at the
moment, and we get the optimal solution to the complete
problem.
Approach: To solve the problem follow the below idea:
The greedy choice is to always pick the next activity whose finish time is
the least among the remaining activities and the start time is more than
or equal to the finish time of the previously selected activity. We can sort
the activities according to their finishing time so that we always consider
the next activity as the minimum finishing time activity
Follow the given steps to solve the problem:
• Sort the activities according to their finishing time
• Select the first activity from the sorted array and print it
• Do the following for the remaining activities in the sorted array
• If the start time of this activity is greater than or equal
to the finish time of the previously selected activity
then select this activity and print it
Note: In the implementation, it is assumed that the activities are already
sorted according to their finish time, otherwise the time complexity will
rise to O(N*log(N)) and Auxiliary Space will rise to O(N), as we have to
create a 2-D array for storing the start and finish times together.
Below is the implementation of the above approach.
ActivitySelection(startTimes, endTimes):
n = length(startTimes)
selectedActivities = []
// Sort activities by their end times
SortActivities(startTimes, endTimes)
// Select the first activity
selectedActivities.append(1)
lastSelected = 1
// Iterate through the remaining activities
for i = 2 to n:
// If the current activity's start time is after the last selected activity's end time,
// then it can be included in the schedule
if startTimes[i] >= endTimes[lastSelected]:
selectedActivities.append(i)
lastSelected = i
return selectedActivities
SortActivities(startTimes, endTimes):
// Sort activities based on their end times
Sort (startTimes, endTimes) based on endTimes
1. Sort the activities based on their end times in non-decreasing order.
2. Select the first activity from the sorted list.
3. For each remaining activity:
If the start time of the current activity is greater than or equal to the end time of the last
selected activity:
Select this activity.
// C++ program for activity selection problem.
// The following implementation assumes that the activities

// are already sorted according to their finish time
#include <bits/stdc++.h>
using namespace std;
// Prints a maximum set of activities that can be done by a

// single person, one at a time.
void printMaxActivities(int s[], int f[], int n)
{
int i, j;
cout << "Following activities are selected" << endl;
// The first activity always gets selected
i = 0;
cout << i << " ";
// Consider rest of the activities

for (j = 1; j < n; j++) {
// If this activity has start time greater than or
// equal to the finish time of previously selected
// activity, then select it
if (s[j] >= f[i]) {
cout << j << " ";
i = j;
}
}
}
// Driver code
int main()
{
int s[] = { 1, 3, 0, 5, 8, 5 };
int f[] = { 2, 4, 6, 7, 9, 9 };
int n = sizeof(s) / sizeof(s[0]);
// Function call
printMaxActivities(s, f, n);
return 0;
}
Output
Following act
Fractional Knapsack Problem

Given the weights and profits of N items, in the form of {profit, weight} put these
items in a knapsack of capacity W to get the maximum total profit in the knapsack.
In Fractional Knapsack, we can break items for maximizing the total value of the
knapsack.
Input: arr[] = {{60, 10}, {100, 20}, {120, 30}}, W = 50
Output: 240
Explanation: By taking items of weight 10 and 20 kg and 2/3 fraction of 30 kg.
Hence total price will be 60+100+(2/3)(120) = 240
Input: arr[] = {{500, 30}}, W = 10
Output: 166.667
Naive Approach: To solve the problem follow the below idea:
Try all possible subsets with all different fractions.
Time Complexity: O(2N)
Auxiliary Space: O(N)
Fractional Knapsack Problem using Greedy algorithm:
An efficient solution is to use the Greedy approach.
The basic idea of the greedy approach is to calculate the
ratio profit/weight for each item and sort the item on the basis of this
ratio. Then take the item with the highest ratio and add them as much as
we can (can be the whole element or a fraction of it).
This will always give the maximum profit because, in each step it adds an
element such that this is the maximum possible profit for that much
weight.
Illustration:
Check the below illustration for a better understanding:
Consider the example: arr[] = {{100, 20}, {60, 10}, {120, 30}}, W = 50.
Sorting: Initially sort the array based on the profit/weight ratio. The
sorted array will be {{60, 10}, {100, 20}, {120, 30}}.
Iteration:
• For i = 0, weight = 10 which is less than W. So add this element
in the knapsack. profit = 60 and remaining W = 50 – 10 = 40.
• For i = 1, weight = 20 which is less than W. So add this element
too. profit = 60 + 100 = 160 and remaining W = 40 – 20 = 20.
• For i = 2, weight = 30 is greater than W. So add 20/30 fraction
= 2/3 fraction of the element. Therefore profit = 2/3 * 120 + 160
= 80 + 160 = 240 and remaining W becomes 0.
So the final profit becomes 240 for W = 50.
Follow the given steps to solve the problem using the above approach:
• Calculate the ratio (profit/weight) for each item.
• Sort all the items in decreasing order of the ratio.
• Initialize res = 0, curr_cap = given_cap.
• Do the following for every item i in the sorted order:
• If the weight of the current item is less than or equal to
the remaining capacity then add the value of that item
into the result
• Else add the current item as much as we can and break
out of the loop.
• Return res.
FractionalKnapsack(items, capacity):
totalValue = 0
currentWeight = 0
// Loop through each item
for each item in items:
// If adding the entire item doesn't exceed capacity, add it fully
if currentWeight + item.weight <= capacity:
totalValue += item.value
currentWeight += item.weight
// Otherwise, add a fraction of the item
else:
remainingWeight = capacity - currentWeight
totalValue += item.value * (remainingWeight / item.weight)
break // Exit loop since capacity is reached
return totalValue
// C++ program to solve fractional Knapsack Problem
// Structure for an item which stores weight and

// corresponding value of Item
struct Item {
int profit, weight;
// Constructor
Item(int profit, int weight)
{
this->profit = profit;
this->weight = weight;
}
};
// Comparison function to sort Item

// according to profit/weight ratio
static bool cmp(struct Item a, struct Item b)
{
double r1 = (double)a.profit / (double)a.weight;
double r2 = (double)b.profit / (double)b.weight;
return r1 > r2;
}
// Main greedy function to solve problem

double fractionalKnapsack(int W, struct Item arr[], int N)
{
// Sorting Item on basis of ratio
sort(arr, arr + N, cmp);
double finalvalue = 0.0;
// Looping through all items

for (int i = 0; i < N; i++) {
// If adding Item won't overflow,

// add it completely
if (arr[i].weight <= W) {
W -= arr[i].weight;
finalvalue += arr[i].profit;
}
// If we can't add current Item,

// add fractional part of it
else {
finalvalue
+= arr[i].profit
* ((double)W / (double)arr[i].weight);
break;
}
}
// Returning final value
return finalvalue;
}
// Driver code
int main()
{
int W = 50;
Item arr[] = { { 60, 10 }, { 100, 20 }, { 120, 30 } };
int N = sizeof(arr) / sizeof(arr[0]);
// Function call
cout << fractionalKnapsack(W, arr, N);
return 0;
}
Huffman Coding
Huffman coding is a lossless data compression algorithm. The idea is to
assign variable-length codes to input characters, lengths of the assigned
codes are based on the frequencies of corresponding characters.
The variable-length codes assigned to input characters are Prefix Codes,
means the codes (bit sequences) are assigned in such a way that the
code assigned to one character is not the prefix of code assigned to any
other character. This is how Huffman Coding makes sure that there is no
ambiguity when decoding the generated bitstream.
Let us understand prefix codes with a counter example. Let there be four
characters a, b, c and d, and their corresponding variable length codes be
00, 01, 0 and 1. This coding leads to ambiguity because code assigned to
c is the prefix of codes assigned to a and b. If the compressed bit stream
is 0001, the de-compressed output may be “cccd” or “ccb” or “acd” or
“ab”.
See this for applications of Huffman Coding.
There are mainly two major parts in Huffman Coding
1. Build a Huffman Tree from input characters.
2. Traverse the Huffman Tree and assign codes to characters.
Algorithm:
The method which is used to construct optimal prefix code is

called Huffman coding.
This algorithm builds a tree in bottom up manner. We can denote this
tree by T
Let, |c| be number of leaves
|c| -1 are number of operations required to merge the nodes. Q be the
priority queue which can be used while constructing binary heap.
Algorithm Huffman (c)
{
n= |c|
Q = c
for i<-1 to n-1
do
{
temp <- get node ()
left (temp] Get_min (Q) right [temp] Get Min (Q)
a = left [templ b = right [temp]
F [temp]<- f[a] + [b]
insert (Q, temp)
return Get_min (0)

}
Steps to build Huffman Tree

Input is an array of unique characters along with their frequency of
occurrences and output is Huffman Tree.
1. Create a leaf node for each unique character and build a min
heap of all leaf nodes (Min Heap is used as a priority queue. The
value of frequency field is used to compare two nodes in min
heap. Initially, the least frequent character is at root)
2. Extract two nodes with the minimum frequency from the min
heap.
3. Create a new internal node with a frequency equal to the sum of

the two nodes frequencies. Make the first extracted node as its
left child and the other extracted node as its right child. Add this
node to the min heap.
4. Repeat steps#2 and #3 until the heap contains only one node.
The remaining node is the root node and the tree is complete.
Let us understand the algorithm with an example:
character Frequency
a 5
b 9
c 12
d 13
e 16
f 45
Step 1. Build a min heap that contains 6 nodes where each node
represents root of a tree with single node.
Step 2 Extract two minimum frequency nodes from min heap. Add a new
internal node with frequency 5 + 9 = 14.
Illustration of step 2
Now min heap contains 5 nodes where 4 nodes are roots of trees with
single element each, and one heap node is root of tree with 3 elements
character Frequency
c 12
d 13
Internal Node 14
e 16
f 45
Step 3: Extract two minimum frequency nodes from heap. Add a new
internal node with frequency 12 + 13 = 25
Now min heap contains 4 nodes where 2 nodes are roots of trees with
single element each, and two heap nodes are root of tree with more than
one nodes
character Frequency
Internal Node 14
e 16
Internal Node 25
f 45
Step 4: Extract two minimum frequency nodes. Add a new internal node
with frequency 14 + 16 = 30
Now min heap contains 3 nodes.

character Frequency
Internal Node 25
Internal Node 30
f 45
Now min heap contains 2 nodes.

character Frequency
f 45
Internal Node 55
Now min heap contains only one node.

character Frequency
Internal Node 100
Since the heap contains only one node, the algorithm stops here.
Steps to print codes from Huffman Tree:
Traverse the tree formed starting from the root. Maintain an auxiliary
array. While moving to the left child, write 0 to the array. While moving
to the right child, write 1 to the array. Print the array when a leaf node is
encountered.
Steps to print code from HuffmanTree
The codes are as follows:

character code-word
f 0
c 100
d 101
a 1100
b 1101
e 111
Huffman(C):
Create a frequency table for characters in C
Create a priority queue Q and initialize it with characters and their frequencies
while Q has more than one element:
x = Extract-Min(Q) // Get the character with the lowest frequency
y = Extract-Min(Q) // Get the character with the second lowest frequency
z = Create a new node with x and y as children, and sum of their frequencies as frequency
Insert(Q, z) // Insert the new node back into the priority queue
return the only remaining node in Q (the root of the Huffman tree)
Extract-Min(Q):
// Remove and return the node with the minimum frequency from priority queue Q
Insert(Q, x):
// Insert node x into priority queue Q
// Append x to Q
Q.append(x)
Sort Q based on frequencies
// A Huffman tree node

struct MinHeapNode {
// One of the input characters

char data;
// Frequency of the character

unsigned freq;
// Left and right child of this node

struct MinHeapNode *left, *right;
};
// A Min Heap: Collection of

// min-heap (or Huffman tree) nodes
struct MinHeap {
// Current size of min heap

unsigned size;
// capacity of min heap

unsigned capacity;
// Array of minheap node pointers

struct MinHeapNode** array;
};
// A utility function allocate a new

// min heap node with given character
// and frequency of the character
struct MinHeapNode* newNode(char data, unsigned freq)
{
struct MinHeapNode* temp = (struct MinHeapNode*)malloc(
sizeof(struct MinHeapNode));
temp->left = temp->right = NULL;

temp->data = data;
temp->freq = freq;
return temp;
}
// A utility function to create

// a min heap of given capacity
struct MinHeap* createMinHeap(unsigned capacity)
struct MinHeap* minHeap

= (struct MinHeap*)malloc(sizeof(struct MinHeap));
// current size is 0
minHeap->size = 0;
minHeap->capacity = capacity;
minHeap->array = (struct MinHeapNode**)malloc(

minHeap->capacity * sizeof(struct MinHeapNode*));
return minHeap;
}
// A utility function to
// swap two min heap nodes
void swapMinHeapNode(struct MinHeapNode** a,
struct MinHeapNode** b)
struct MinHeapNode* t = *a;

*a = *b;
*b = t;
}
// The standard minHeapify function.

void minHeapify(struct MinHeap* minHeap, int idx)
int smallest = idx;

int left = 2 * idx + 1;
int right = 2 * idx + 2;
if (left < minHeap->size

&& minHeap->array[left]->freq
< minHeap->array[smallest]->freq)
smallest = left;
if (right < minHeap->size

&& minHeap->array[right]->freq
< minHeap->array[smallest]->freq)
smallest = right;
if (smallest != idx) {
swapMinHeapNode(&minHeap->array[smallest],
&minHeap->array[idx]);
minHeapify(minHeap, smallest);
}
}
// A utility function to check

// if size of heap is 1 or not
int isSizeOne(struct MinHeap* minHeap)
{
return (minHeap->size == 1);

}
// A standard function to extract

// minimum value node from heap
struct MinHeapNode* extractMin(struct MinHeap* minHeap)
struct MinHeapNode* temp = minHeap->array[0];

minHeap->array[0] = minHeap->array[minHeap->size - 1];
--minHeap->size;
minHeapify(minHeap, 0);
return temp;
}
// A utility function to insert

// a new node to Min Heap
void insertMinHeap(struct MinHeap* minHeap,
struct MinHeapNode* minHeapNode)
++minHeap->size;
int i = minHeap->size - 1;
while (i
&& minHeapNode->freq
< minHeap->array[(i - 1) / 2]->freq) {
minHeap->array[i] = minHeap->array[(i - 1) / 2];

i = (i - 1) / 2;
}
minHeap->array[i] = minHeapNode;
}
// A standard function to build min heap

void buildMinHeap(struct MinHeap* minHeap)
int n = minHeap->size - 1;
int i;
for (i = (n - 1) / 2; i >= 0; --i)

minHeapify(minHeap, i);
}
// A utility function to print an array of size n

void printArr(int arr[], int n)
{
int i;
for (i = 0; i < n; ++i)
cout << arr[i];
cout << "\n";

}
// Utility function to check if this node is leaf

int isLeaf(struct MinHeapNode* root)
return !(root->left) && !(root->right);

}
// Creates a min heap of capacity

// equal to size and inserts all character of
// data[] in min heap. Initially size of
// min heap is equal to capacity
struct MinHeap* createAndBuildMinHeap(char data[],
int freq[], int size)
struct MinHeap* minHeap = createMinHeap(size);
for (int i = 0; i < size; ++i)

minHeap->array[i] = newNode(data[i], freq[i]);
minHeap->size = size;
buildMinHeap(minHeap);
return minHeap;
}
// The main function that builds Huffman tree

struct MinHeapNode* buildHuffmanTree(char data[],
int freq[], int size)
{
struct MinHeapNode *left, *right, *top;
// Step 1: Create a min heap of capacity

// equal to size. Initially, there are
// modes equal to size.
struct MinHeap* minHeap
= createAndBuildMinHeap(data, freq, size);
// Iterate while size of heap doesn't become 1

while (!isSizeOne(minHeap)) {
// Step 2: Extract the two minimum

// freq items from min heap
left = extractMin(minHeap);
right = extractMin(minHeap);
// Step 3: Create a new internal

// node with frequency equal to the
// sum of the two nodes frequencies.
// Make the two extracted node as
// left and right children of this new node.
// Add this node to the min heap
// '$' is a special value for internal nodes, not
// used
top = newNode('$', left->freq + right->freq);
top->left = left;
top->right = right;
insertMinHeap(minHeap, top);
}
// Step 4: The remaining node is the

// root node and the tree is complete.
return extractMin(minHeap);
}
// Prints huffman codes from the root of Huffman Tree.
// It uses arr[] to store codes
void printCodes(struct MinHeapNode* root, int arr[],
int top)
// Assign 0 to left edge and recur

if (root->left) {
arr[top] = 0;
printCodes(root->left, arr, top + 1);
}
// Assign 1 to right edge and recur

if (root->right) {
arr[top] = 1;
printCodes(root->right, arr, top + 1);
}
// If this is a leaf node, then

// it contains one of the input
// characters, print the character
// and its code from arr[]
if (isLeaf(root)) {
cout << root->data << ": ";

printArr(arr, top);
}
}
// The main function that builds a

// Huffman Tree and print codes by traversing
// the built Huffman Tree
void HuffmanCodes(char data[], int freq[], int size)
{
// Construct Huffman Tree
struct MinHeapNode* root
= buildHuffmanTree(data, freq, size);
// Print Huffman codes using

// the Huffman tree built above
int arr[MAX_TREE_HT], top = 0;
printCodes(root, arr, top);

}
// Driver code
int main()
{
char arr[] = { 'a', 'b', 'c', 'd', 'e', 'f' };

int freq[] = { 5, 9, 12, 13, 16, 45 };
int size = sizeof(arr) / sizeof(arr[0]);
HuffmanCodes(arr, freq, size);
return 0;
}
Kruskal’s Minimum Spanning Tree (MST)

Algorithm
A minimum spanning tree (MST) or minimum weight spanning tree for a
weighted, connected, undirected graph is a spanning tree with a weight
less than or equal to the weight of every other spanning tree. To learn
more about Minimum Spanning Tree, refer to this article.
Introduction to Kruskal’s Algorithm:
Here we will discuss Kruskal’s algorithm to find the MST of a given
weighted graph.
In Kruskal’s algorithm, sort all edges of the given graph in increasing
order. Then it keeps on adding new edges and nodes in the MST if the
newly added edge does not form a cycle. It picks the minimum weighted
edge at first and the maximum weighted edge at last. Thus we can say
that it makes a locally optimal choice in each step in order to find the
optimal solution. Hence this is a Greedy Algorithm.
How to find MST using Kruskal’s algorithm?
Below are the steps for finding MST using Kruskal’s algorithm:
1. Sort all the edges in non-decreasing order of their weight.
2. Pick the smallest edge. Check if it forms a cycle with the
spanning tree formed so far. If the cycle is not formed, include
this edge. Else, discard it.
3. Repeat step#2 until there are (V-1) edges in the spanning tree.
Step 2 uses the Union-Find algorithm to detect cycles.
Kruskal’s algorithm to find the minimum cost spanning tree uses the
greedy approach. The Greedy Choice is to pick the smallest weight edge
that does not cause a cycle in the MST constructed so far. Let us
understand it with an example:
Illustration:
Below is the illustration of the above approach:
Input Graph:
The graph contains 9 vertices and 14 edges. So, the minimum spanning
tree formed will be having (9 – 1) = 8 edges.
After sorting:
Weight Source Destination
1 7 6
2 8 2
2 6 5
4 0 1
4 2 5
6 8 6
7 2 3
Weight Source Destination
7 7 8
8 0 7
8 1 2
9 3 4
10 5 4
11 1 7
14 3 5
Now pick all edges one by one from the sorted list of edges
Step 1: Pick edge 7-6. No cycle is formed, include it.
Add edge 7-6 in the MST


Step 6: Pick edge 8-6. Since including this edge results in the cycle,
discard it. Pick edge 2-3: No cycle is formed, include it.
discard it. Pick edge 0-7. No cycle is formed, include it.
Add edge 0-7 in MST
discard it. Pick edge 3-4. No cycle is formed, include it.
Note: Since the number of edges included in the MST equals to (V – 1),
so the algorithm stops here
Below is the implementation of the above approach:
/ C++ program for the above approach
// DSU data structure

// path compression + rank by union
class DSU {
int* parent;
int* rank;
public:
DSU(int n)
{
parent = new int[n];
rank = new int[n];
for (int i = 0; i < n; i++) {
parent[i] = -1;
rank[i] = 1;
}
}
// Find function
int find(int i)
{
if (parent[i] == -1)
return i;
return parent[i] = find(parent[i]);

}
// Union function
void unite(int x, int y)
{
int s1 = find(x);
int s2 = find(y);
if (s1 != s2) {
if (rank[s1] < rank[s2]) {
parent[s1] = s2;
}
else if (rank[s1] > rank[s2]) {
parent[s2] = s1;
}
else {
parent[s2] = s1;
rank[s1] += 1;
}
}
}
};
class Graph {
vector<vector<int> > edgelist;
int V;
public:
Graph(int V) { this->V = V; }
// Function to add edge in a graph

void addEdge(int x, int y, int w)
{
edgelist.push_back({ w, x, y });
}
void kruskals_mst()
{
// Sort all edges
sort(edgelist.begin(), edgelist.end());
// Initialize the DSU

DSU s(V);
int ans = 0;
cout << "Following are the edges in the "
"constructed MST"
<< endl;
for (auto edge : edgelist) {
int w = edge[0];
int x = edge[1];
int y = edge[2];
// Take this edge in MST if it does

// not forms a cycle
if (s.find(x) != s.find(y)) {
s.unite(x, y);
ans += w;
cout << x << " -- " << y << " == " << w
<< endl;
}
}
cout << "Minimum Cost Spanning Tree: " << ans;
}
};
// Driver code
int main()
{
Graph g(4);
g.addEdge(0, 1, 10);
g.addEdge(2, 3, 4);
g.addEdge(2, 0, 6);
g.addEdge(0, 3, 5);
// Function call
g.kruskals_mst();
return 0;
}
Kruskal(G):
Initialize an empty set A to store the edges of the minimum spanning tree
Sort the edges of G in non-decreasing order of their weights
Initialize a disjoint-set data structure to keep track of connected components
for each vertex v in G:

Make-Set(v) // Create a set for each vertex
for each edge (u, v) in sorted order of G:

if Find(u) ≠ Find(v): // Check if including this edge forms a cycle
Add (u, v) to A // Include the edge in the minimum spanning tree
Union(u, v) // Merge the sets containing u and v
return A // Return the set of edges of the minimum spanning tree
Find(u):
// Find the representative of the set containing u
if u ≠ parent[u]:
parent[u] = Find(parent[u])
return parent[u]
Union(u, v):
// Merge the sets containing u and v
root_u = Find(u)
root_v = Find(v)
parent[root_u] = root_v
Prim’s Algorithm for Minimum Spanning Tree

(MST)
Introduction to Prim’s algorithm:
We have discussed Kruskal’s algorithm for Minimum Spanning Tree. Like
Kruskal’s algorithm, Prim’s algorithm is also a Greedy algorithm. This
algorithm always starts with a single node and moves through several
adjacent nodes, in order to explore all of the connected edges along the
way.
The algorithm starts with an empty spanning tree. The idea is to maintain
two sets of vertices. The first set contains the vertices already included in
the MST, and the other set contains the vertices not yet included. At
every step, it considers all the edges that connect the two sets and picks
the minimum weight edge from these edges. After picking the edge, it
moves the other endpoint of the edge to the set containing MST.
A group of edges that connects two sets of vertices in a graph is
called cut in graph theory. So, at every step of Prim’s algorithm, find a
cut, pick the minimum weight edge from the cut, and include this vertex
in MST Set (the set that contains already included vertices).
How does Prim’s Algorithm Work?
The working of Prim’s algorithm can be described by using the following
steps:
Note: For determining a cycle, we can divide the vertices into two sets
[one set contains the vertices included in MST and the other contains the
fringe vertices.]
Illustration of Prim’s Algorithm:

Consider the following graph as an example for which we need to find
the Minimum Spanning Tree (MST).
Example of a graph
Step 1: Firstly, we select an arbitrary vertex that acts as the starting
vertex of the Minimum Spanning Tree. Here we have selected vertex 0 as
the starting vertex.
0 is selected as starting vertex
Step 2: All the edges connecting the incomplete MST and other vertices
are the edges {0, 1} and {0, 7}. Between these two the edge with
minimum weight is {0, 1}. So include the edge and vertex 1 in the MST.
1 is added to the MST

Step 3: The edges connecting the incomplete MST to other vertices are
{0, 7}, {1, 7} and {1, 2}. Among these edges the minimum weight is 8
which is of the edges {0, 7} and {1, 2}. Let us here include the edge {0, 7}
and the vertex 7 in the MST. [We could have also included edge {1, 2}
and vertex 2 in the MST].
7 is added in the MST
Step 4: The edges that connect the incomplete MST with the fringe
vertices are {1, 2}, {7, 6} and {7, 8}. Add the edge {7, 6} and the vertex 6
in the MST as it has the least weight (i.e., 1).
6 is added in the MST
Step 5: The connecting edges now are {7, 8}, {1, 2}, {6, 8} and {6, 5}.
Include edge {6, 5} and vertex 5 in the MST as the edge has the minimum
weight (i.e., 2) among them.
Include vertex 5 in the MST

Step 6: Among the current connecting edges, the edge {5, 2} has the
minimum weight. So include that edge and the vertex 2 in the MST.
Step 7: The connecting edges between the incomplete MST and the
other edges are {2, 8}, {2, 3}, {5, 3} and {5, 4}. The edge with minimum
weight is edge {2, 8} which has weight 2. So include this edge and the
vertex 8 in the MST.
Add vertex 8 in the MST

Step 8: See here that the edges {7, 8} and {2, 3} both have same weight
which are minimum. But 7 is already part of MST. So we will consider the
edge {2, 3} and include that edge and vertex 3 in the MST.
Include vertex 3 in MST
Step 9: Only the vertex 4 remains to be included. The minimum

weighted edge from the incomplete MST to 4 is {3, 4}.

The final structure of the MST is as follows and the weight of the edges
of the MST is (4 + 8 + 1 + 2 + 4 + 2 + 7 + 9) = 37.
The structure of the MST formed using the above method
Note: If we had selected the edge {1, 2} in the third step then the MST
would look like the following.
Structure of the alternate MST if we had selected edge {1, 2} in the MST
How to implement Prim’s Algorithm?
Follow the given steps to utilize the Prim’s Algorithm mentioned above
for finding MST of a graph:
• Create a set mstSet that keeps track of vertices already included
in MST.
• Assign a key value to all vertices in the input graph. Initialize all
key values as INFINITE. Assign the key value as 0 for the first
vertex so that it is picked first.
• While mstSet doesn’t include all vertices
• Pick a vertex u that is not there in mstSet and has a
minimum key value.
• Include u in the mstSet.
• Update the key value of all adjacent vertices of u. To
update the key values, iterate through all adjacent
vertices.
• For every adjacent vertex v, if the weight of
edge u-v is less than the previous key value
of v, update the key value as the weight of u-v.
The idea of using key values is to pick the minimum weight edge from
the cut. The key values are used only for vertices that are not yet
included in MST, the key value for these vertices indicates the minimum
weight edges connecting them to the set of vertices included in MST.
Below is the implementation of the approach:
// A C++ program for Prim's Minimum

// Spanning Tree (MST) algorithm. The program is
// for adjacency matrix representation of the graph
// Number of vertices in the graph

#define V 5
// A utility function to find the vertex with

// minimum key value, from the set of vertices
// not yet included in MST
int minKey(int key[], bool mstSet[])
{
// Initialize min value
int min = INT_MAX, min_index;
for (int v = 0; v < V; v++)
if (mstSet[v] == false && key[v] < min)
min = key[v], min_index = v;
return min_index;
}
// A utility function to print the

// constructed MST stored in parent[]
void printMST(int parent[], int graph[V][V])
{
cout << "Edge \tWeight\n";
for (int i = 1; i < V; i++)
cout << parent[i] << " - " << i << " \t"
<< graph[i][parent[i]] << " \n";
}
// Function to construct and print MST for

// a graph represented using adjacency
// matrix representation
void primMST(int graph[V][V])
{
// Array to store constructed MST
int parent[V];
// Key values used to pick minimum weight edge in cut

int key[V];
// To represent set of vertices included in MST

bool mstSet[V];
// Initialize all keys as INFINITE

for (int i = 0; i < V; i++)
key[i] = INT_MAX, mstSet[i] = false;
// Always include first 1st vertex in MST.

// Make key 0 so that this vertex is picked as first
// vertex.
key[0] = 0;
// First node is always root of MST

parent[0] = -1;
// The MST will have V vertices

for (int count = 0; count < V - 1; count++) {
// Pick the minimum key vertex from the

// set of vertices not yet included in MST
int u = minKey(key, mstSet);
// Add the picked vertex to the MST Set

mstSet[u] = true;
// Update key value and parent index of

// the adjacent vertices of the picked vertex.
// Consider only those vertices which are not
// yet included in MST
for (int v = 0; v < V; v++)
// graph[u][v] is non zero only for adjacent

// vertices of m mstSet[v] is false for vertices
// not yet included in MST Update the key only
// if graph[u][v] is smaller than key[v]
if (graph[u][v] && mstSet[v] == false
&& graph[u][v] < key[v])
parent[v] = u, key[v] = graph[u][v];
}
// Print the constructed MST

printMST(parent, graph);
}
// Driver's code
int main()
{
int graph[V][V] = { { 0, 2, 0, 6, 0 },
{ 2, 0, 3, 8, 5 },
{ 0, 3, 0, 0, 7 },
{ 6, 8, 0, 0, 9 },
{ 0, 5, 7, 9, 0 } };
// Print the solution

primMST(graph);
return 0;
}
Output
Edge Weight
0 - 1 2
1 - 2 3
0 - 3 6
1 - 4 5
Complexity Analysis of Prim’s Algorithm:
Time Complexity: O(V2), If the input graph is represented using an
adjacency list, then the time complexity of Prim’s algorithm can be
reduced to O(E * logV) with the help of a binary heap. In this
implementation, we are always considering the spanning tree to start
from the root of the graph
Auxiliary Space: O(V)
Bellman–Ford Algorithm
Imagine you have a map with different cities connected by roads, each road
having a certain distance. The Bellman–Ford algorithm is like a guide that
helps you find the shortest path from one city to all other cities, even if some
roads have negative lengths. It’s like a GPS for computers, useful for figuring
out the quickest way to get from one point to another in a network. In this
article, we’ll take a closer look at how this algorithm works and why it’s so
handy in solving everyday problems.
Bellman-Ford Algorithm
Bellman-Ford is a single source shortest path algorithm that determines
the shortest path between a given source vertex and every other vertex
in a graph. This algorithm can be used on
both weighted and unweighted graphs.
A Bellman-Ford algorithm is also guaranteed to find the shortest path in
a graph, similar to Dijkstra’s algorithm. Although Bellman-Ford is slower
than Dijkstra’s algorithm, it is capable of handling graphs with negative
edge weights, which makes it more versatile. The shortest path cannot
be found if there exists a negative cycle in the graph. If we continue to go
around the negative cycle an infinite number of times, then the cost of
the path will continue to decrease (even though the length of the path is
increasing). As a result, Bellman-Ford is also capable of
detecting negative cycles, which is an important feature.
The idea behind Bellman Ford Algorithm:
The Bellman-Ford algorithm’s primary principle is that it starts with a
single source and calculates the distance to each node. The distance is
initially unknown and assumed to be infinite, but as time goes on, the
algorithm relaxes those paths by identifying a few shorter paths. Hence it
is said that Bellman-Ford is based on “Principle of Relaxation“.
Principle of Relaxation of Edges for Bellman-Ford:
• It states that for the graph having N vertices, all the edges
should be relaxed N-1 times to compute the single source
shortest path.
• In order to detect whether a negative cycle exists or not, relax
all the edge one more time and if the shortest distance for any
node reduces then we can say that a negative cycle exists. In
short if we relax the edges N times, and there is any change in
the shortest distance of any node between the N-
1th and Nth relaxation than a negative cycle exists, otherwise
not exist.
Why Relaxing Edges N-1 times, gives us Single Source Shortest Path?
In the worst-case scenario, a shortest path between two vertices can
have at most N-1 edges, where N is the number of vertices. This is
because a simple path in a graph with N vertices can have at most N-
1 edges, as it’s impossible to form a closed loop without revisiting a
vertex.
By relaxing edges N-1 times, the Bellman-Ford algorithm ensures that
the distance estimates for all vertices have been updated to their optimal
values, assuming the graph doesn’t contain any negative-weight cycles
reachable from the source vertex. If a graph contains a negative-weight
cycle reachable from the source vertex, the algorithm can detect it
after N-1 iterations, since the negative cycle disrupts the shortest path
lengths.
In summary, relaxing edges N-1 times in the Bellman-Ford algorithm
guarantees that the algorithm has explored all possible paths of length
up to N-1, which is the maximum possible length of a shortest path in a
graph with N vertices. This allows the algorithm to correctly calculate the
shortest paths from the source vertex to all other vertices, given that
there are no negative-weight cycles.
Why Does the Reduction of Distance in the N’th Relaxation Indicates the
Existence of a Negative Cycle?
As previously discussed, achieving the single source shortest paths to all
other nodes takes N-1 relaxations. If the N’th relaxation further reduces
the shortest distance for any node, it implies that a certain edge with
negative weight has been traversed once more. It is important to note
that during the N-1 relaxations, we presumed that each vertex is
traversed only once. However, the reduction of distance during the N’th
relaxation indicates revisiting a vertex.
Working of Bellman-Ford Algorithm to Detect the Negative cycle in the
graph:
Let’s suppose we have a graph which is given below and we want to find
whether there exists a negative cycle or not using Bellman-Ford.
Initial Graph
Step 1: Initialize a distance array Dist[] to store the shortest distance for
each vertex from the source vertex. Initially distance of source will be 0
and Distance of other vertices will be INFINITY.
Initialize a distance array
Step 2: Start relaxing the edges, during 1st Relaxation:

• Current Distance of B > (Distance of A) + (Weight of A to B) i.e.
Infinity > 0 + 5
• Therefore, Dist[B] = 5
1st Relaxation
Step 3: During 2nd Relaxation:
• Current Distance of D > (Distance of B) + (Weight of B to D) i.e.

Infinity > 5 + 2
• Dist[D] = 7
• Current Distance of C > (Distance of B) + (Weight of B to C) i.e.
Infinity > 5 + 1
• Dist[C] = 6
2nd Relaxation
Step 4: During 3rd Relaxation:

• Current Distance of F > (Distance of D ) + (Weight of D to F) i.e.
Infinity > 7 + 2
• Dist[F] = 9
• Current Distance of E > (Distance of C ) + (Weight of C to E) i.e.
Infinity > 6 + 1
• Dist[E] = 7
3rd Relaxation
Step 5: During 4th Relaxation:

• Current Distance of D > (Distance of E) + (Weight of E to D) i.e. 7
> 7 + (-1)
• Dist[D] = 6
• Current Distance of E > (Distance of F ) + (Weight of F to E) i.e. 7
> 9 + (-3)
• Dist[E] = 6
4th Relaxation
Step 6: During 5th Relaxation:

• Current Distance of F > (Distance of D) + (Weight of D to F) i.e. 9
>6+2
• Dist[F] = 8
• Current Distance of D > (Distance of E ) + (Weight of E to D) i.e.
6 > 6 + (-1)
• Dist[D] = 5
• Since the graph h 6 vertices, So during the 5th relaxation the
shortest distance for all the vertices should have been
calculated.
5th Relaxation
Step 7: Now the final relaxation i.e. the 6th relaxation should indicate the
presence of negative cycle if there is any changes in the distance array of
5th relaxation.
During the 6th relaxation, following changes can be seen:
• Current Distance of E > (Distance of F) + (Weight of F to E) i.e. 6
> 8 + (-3)
• Dist[E]=5
• Current Distance of F > (Distance of D ) + (Weight of D to F) i.e.
8>5+2
• Dist[F]=7
Since, we observer changes in the Distance array Hence ,we can
conclude the presence of a negative cycle in the graph.
6th Relaxation
Result: A negative cycle (D->F->E) exists in the graph.

Algorithm to Find Negative Cycle in a Directed Weighted Graph Using
Bellman-Ford:
• Initialize distance array dist[] for each vertex ‘v‘ as dist[v] =
INFINITY.
• Assume any vertex (let’s say ‘0’) as source and assign dist = 0.
• Relax all the edges(u,v,weight) N-1 times as per the below
condition:
• dist[v] = minimum(dist[v], distance[u] + weight)
• Now, Relax all the edges one more time i.e. the Nth time and
based on the below two cases we can detect the negative cycle:
• Case 1 (Negative cycle exists): For any edge(u, v,
weight), if dist[u] + weight < dist[v]
• Case 2 (No Negative cycle) : case 1 fails for all the
edges.
Handling Disconnected Graphs in the Algorithm:
The above algorithm and program might not work if the given graph is
disconnected. It works when all vertices are reachable from source
vertex 0.
To handle disconnected graphs, we can repeat the above algorithm for
vertices having distance = INFINITY, or simply for the vertices that are
not visited.
// A C++ program for Bellman-Ford's single source
// shortest path algorithm.
// a structure to represent a weighted edge in graph

struct Edge {
int src, dest, weight;
};
// a structure to represent a connected, directed and

// weighted graph
struct Graph {
// V-> Number of vertices, E-> Number of edges
int V, E;
// graph is represented as an array of edges.

struct Edge* edge;
};
// Creates a graph with V vertices and E edges

struct Graph* createGraph(int V, int E)
{
struct Graph* graph = new Graph;
graph->V = V;
graph->E = E;
graph->edge = new Edge[E];
return graph;
}
// A utility function used to print the solution

void printArr(int dist[], int n)
{
printf("Vertex Distance from Source\n");
for (int i = 0; i < n; ++i)
printf("%d \t\t %d\n", i, dist[i]);
}
// The main function that finds shortest distances from src

// to all other vertices using Bellman-Ford algorithm. The
// function also detects negative weight cycle
void BellmanFord(struct Graph* graph, int src)
{
int V = graph->V;
int E = graph->E;
int dist[V];
// Step 1: Initialize distances from src to all other

// vertices as INFINITE
for (int i = 0; i < V; i++)
dist[i] = INT_MAX;
dist[src] = 0;
// Step 2: Relax all edges |V| - 1 times. A simple
// shortest path from src to any other vertex can have
// at-most |V| - 1 edges
for (int i = 1; i <= V - 1; i++) {
for (int j = 0; j < E; j++) {
int u = graph->edge[j].src;
int v = graph->edge[j].dest;
int weight = graph->edge[j].weight;
if (dist[u] != INT_MAX
&& dist[u] + weight < dist[v])
dist[v] = dist[u] + weight;
}
}
// Step 3: check for negative-weight cycles. The above

// step guarantees shortest distances if graph doesn't
// contain negative weight cycle. If we get a shorter
// path, then there is a cycle.
for (int i = 0; i < E; i++) {
int u = graph->edge[i].src;
int v = graph->edge[i].dest;
int weight = graph->edge[i].weight;
if (dist[u] != INT_MAX
&& dist[u] + weight < dist[v]) {
printf("Graph contains negative weight cycle");
return; // If negative cycle is detected, simply
// return
}
}
printArr(dist, V);
return;
}
// Driver's code
int main()
{
/* Let us create the graph given in above example */
int V = 5; // Number of vertices in graph
int E = 8; // Number of edges in graph
struct Graph* graph = createGraph(V, E);
// add edge 0-1 (or A-B in above figure)

graph->edge[0].src = 0;
graph->edge[0].dest = 1;
graph->edge[0].weight = -1;
// add edge 0-2 (or A-C in above figure)

graph->edge[1].weight = 4;
// add edge 1-2 (or B-C in above figure)

// add edge 1-3 (or B-D in above figure)

// add edge 1-4 (or B-E in above figure)

// add edge 3-2 (or D-C in above figure)

// add edge 3-1 (or D-B in above figure)

// add edge 4-3 (or E-D in above figure)

graph->edge[7].weight = -3;
// Function call
BellmanFord(graph, 0);
return 0;
}
Output
Vertex Distance from Source
0 0
1 -1
2 2
3 -2
4 1
Complexity Analysis of Bellman-Ford Algorithm:
• Time Complexity when graph is connected:
• Best Case: O(E), when distance array after 1st and 2nd
relaxation are same , we can simply stop further
processing
• Average Case: O(V*E)
• Worst Case: O(V*E)
• Time Complexity when graph is disconnected:
• All the cases: O(E*(V^2))
• Auxiliary Space: O(V), where V is the number of vertices in the
graph.
Bellman Ford’s Algorithm Applications:
• Network Routing: Bellman-Ford is used in computer networking
to find the shortest paths in routing tables, helping data packets
navigate efficiently across networks.
• GPS Navigation: GPS devices use Bellman-Ford to calculate the
shortest or fastest routes between locations, aiding navigation
apps and devices.
• Transportation and Logistics: Bellman-Ford’s algorithm can be
applied to determine the optimal paths for vehicles in
transportation and logistics, minimizing fuel consumption and
travel time.
• Game Development: Bellman-Ford can be used to model
movement and navigation within virtual worlds in game
development, where different paths may have varying costs or
obstacles.
• Robotics and Autonomous Vehicles: The algorithm aids in path
planning for robots or autonomous vehicles, considering
obstacles, terrain, and energy consumption.
Drawback of Bellman Ford’s Algorithm:
• Bellman-Ford algorithm will fail if the graph contains any
negative edge cycle.
What is Dijkstra’s Algorithm? | Introduction
to Dijkstra’s Shortest Path Algorithm
Dijkstra’s Algorithm:
Dijkstra’s algorithm is a popular algorithms for solving many single-
source shortest path problems having non-negative edge weight in the
graphs i.e., it is to find the shortest distance between two vertices on a
graph. It was conceived by Dutch computer scientist Edsger W.
Dijkstra in 1956.
The algorithm maintains a set of visited vertices and a set of unvisited
vertices. It starts at the source vertex and iteratively selects the unvisited
vertex with the smallest tentative distance from the source. It then visits
the neighbors of this vertex and updates their tentative distances if a
shorter path is found. This process continues until the destination vertex
is reached, or all reachable vertices have been visited.
Need for Dijkstra’s Algorithm (Purpose and Use-Cases)
The need for Dijkstra’s algorithm arises in many applications where
finding the shortest path between two points is crucial.
For example, It can be used in the routing protocols for computer
networks and also used by map systems to find the shortest path
between starting point and the Destination (as explained in How does
Google Maps work?)
Can Dijkstra’s Algorithm work on both Directed and Undirected graphs?
Yes, Dijkstra’s algorithm can work on both directed graphs and
undirected graphs as this algorithm is designed to work on any type of
graph as long as it meets the requirements of having non-negative edge
weights and being connected.
• In a directed graph, each edge has a direction, indicating the
direction of travel between the vertices connected by the edge.
In this case, the algorithm follows the direction of the edges
when searching for the shortest path.
• In an undirected graph, the edges have no direction, and the
algorithm can traverse both forward and backward along the
edges when searching for the shortest path.
Algorithm for Dijkstra’s Algorithm:
1. Mark the source node with a current distance of 0 and the rest
with infinity.
2. Set the non-visited node with the smallest current distance as
the current node.
3. For each neighbor, N of the current node adds the current
distance of the adjacent node with the weight of the edge
connecting 0->1. If it is smaller than the current distance of
Node, set it as the new current distance of N.
4. Mark the current node 1 as visited.
5. Go to step 2 if there are any nodes are unvisited.
How does Dijkstra’s Algorithm works?
Let’s see how Dijkstra’s Algorithm works with an example given below:
Dijkstra’s Algorithm will generate the shortest path from Node 0 to all
other Nodes in the graph.
Consider the below graph:
Dijkstra’s Algorithm
The algorithm will generate the shortest path from node 0 to all the
other nodes in the graph.
For this graph, we will assume that the weight of the edges
represents the distance between two nodes.
As, we can see we have the shortest path from,
Node 0 to Node 1, from
Node 0 to Node 6.
Initially we have a set of resources given below :
• The Distance from the source node to itself is 0. In this example
the source node is 0.
• The distance from the source node to all other node is unknown
so we mark all of them as infinity.
Example: 0 -> 0, 1-> ∞,2-> ∞,3-> ∞,4-> ∞,5-> ∞,6-> ∞.
• we’ll also have an array of unvisited elements that will keep
track of unvisited or unmarked Nodes.
• Algorithm will complete when all the nodes marked as visited
and the distance between them added to the path. Unvisited
Nodes:- 0 1 2 3 4 5 6.
Step 1: Start from Node 0 and mark Node as visited as you can check in
below image visited Node is marked red.
Step 2: Check for adjacent Nodes, Now we have to choices (Either

choose Node1 with distance 2 or either choose Node 2 with distance 6 )
and choose Node with minimum distance. In this step Node 1 is Minimum
distance adjacent Node, so marked it as visited and add up the distance.
Distance: Node 0 -> Node 1 = 2
Step 3: Then Move Forward and check for adjacent Node which is Node
3, so marked it as visited and add up the distance, Now the distance will
be:
Distance: Node 0 -> Node 1 -> Node 3 = 2 + 5 = 7
Step 4: Again we have two choices for adjacent Nodes (Either we can
choose Node 4 with distance 10 or either we can choose Node 5 with
distance 15) so choose Node with minimum distance. In this step Node
4 is Minimum distance adjacent Node, so marked it as visited and add up
the distance.
Distance: Node 0 -> Node 1 -> Node 3 -> Node 4 = 2 + 5 + 10 = 17
Step 5: Again, Move Forward and check for adjacent Node which
is Node 6, so marked it as visited and add up the distance, Now the
distance will be:
Distance: Node 0 -> Node 1 -> Node 3 -> Node 4 -> Node 6 = 2 + 5 +
10 + 2 = 19
So, the Shortest Distance from the Source Vertex is 19 which is
optimal one
Pseudo Code for Dijkstra’s Algorithm
function Dijkstra(Graph, source):
// Initialize distances to all nodes as infinity, except for the source node.
distances = map infinity to all nodes
distances = 0
// Initialize an empty set of visited nodes and a priority queue to keep
track of the nodes to visit.
visited = empty set
queue = new PriorityQueue()
queue.enqueue(source, 0)
// Loop until all nodes have been visited.
while queue is not empty:
// Dequeue the node with the smallest distance from the priority
queue.
current = queue.dequeue()
// If the node has already been visited, skip it.
if current in visited:
continue
// Mark the node as visited.
visited.add(current)
// Check all neighboring nodes to see if their distances need to be
updated.
for neighbor in Graph.neighbors(current):
// Calculate the tentative distance to the neighbor through the
current node.
tentative_distance = distances[current] +
Graph.distance(current, neighbor)
// If the tentative distance is smaller than the current distance to
the neighbor, update the distance.
if tentative_distance < distances[neighbor]:
distances[neighbor] = tentative_distance
// Enqueue the neighbor with its new distance to be considered
for visitation in the future.
queue.enqueue(neighbor, distances[neighbor])
// Return the calculated distances from the source to all other nodes in
the graph.
return distances
Implemention of Dijkstra’s Algorithm:

There are several ways to Implement Dijkstra’s algorithm, but the most
common ones are:
1. Priority Queue (Heap-based Implementation):
2. Array-based Implementation:
1. Dijkstra’s Shortest Path Algorithm using priority_queue (Heap)
In this Implementation, we are given a graph and a source vertex in the
graph, finding the shortest paths from the source to all vertices in the
given graph.
Example:
Input: Source = 0
Example
Output: Vertex
Vertex Distance from Source
0 -> 0
1 -> 2
2 -> 6
3 -> 7
4 -> 17
5 -> 22
6 -> 19
Below is the algorithm based on the above idea:
• Initialize the distance values and priority queue.
• Insert the source node into the priority queue with distance 0.
• While the priority queue is not empty:
• Extract the node with the minimum distance from the
priority queue.
• Update the distances of its neighbors if a shorter path
is found.
• Insert updated neighbors into the priority queue.
Below is the C++ Implementation of the above approach:
// Program to find Dijkstra's shortest path using
// priority_queue in STL
#define INF 0x3f3f3f3f
// iPair ==> Integer Pair
typedef pair<int, int> iPair;
// This class represents a directed graph using
// adjacency list representation
class Graph {
int V; // No. of vertices
// In a weighted graph, we need to store vertex
// and weight pair for every edge
list<pair<int, int> >* adj;
public:
Graph(int V); // Constructor
// function to add an edge to graph
void addEdge(int u, int v, int w);

// prints shortest path from s
void shortestPath(int s);
};
// Allocates memory for adjacency list
Graph::Graph(int V)
this->V = V;
adj = new list<iPair>[V];
void Graph::addEdge(int u, int v, int w)
adj[u].push_back(make_pair(v, w));
adj[v].push_back(make_pair(u, w));
// Prints shortest paths from src to all other vertices
void Graph::shortestPath(int src)
// Create a priority queue to store vertices that
// are being preprocessed. This is weird syntax in C++.
// Refer below link for details of this syntax
// https://www.geeksforgeeks.org/implement-min-heap-using-stl/
priority_queue<iPair, vector<iPair>, greater<iPair> >
pq;
// Create a vector for distances and initialize all
// distances as infinite (INF)
vector<int> dist(V, INF);

// Insert source itself in priority queue and initialize
// its distance as 0.
pq.push(make_pair(0, src));
dist[src] = 0;
/* Looping till priority queue becomes empty (or all
distances are not finalized) */
while (!pq.empty()) {
// The first vertex in pair is the minimum distance
// vertex, extract it from priority queue.
// vertex label is stored in second of pair (it
// has to be done this way to keep the vertices
// sorted distance (distance must be first item
// in pair)
int u = pq.top().second;
pq.pop();
// 'i' is used to get all adjacent vertices of a
// vertex
list<pair<int, int> >::iterator i;
for (i = adj[u].begin(); i != adj[u].end(); ++i) {
// Get vertex label and weight of current
// adjacent of u.
int v = (*i).first;
int weight = (*i).second;
// If there is shorted path to v through u.
if (dist[v] > dist[u] + weight) {
// Updating distance of v
dist[v] = dist[u] + weight;

pq.push(make_pair(dist[v], v));
// Print shortest distances stored in dist[]
printf("Vertex Distance from Source\n");
for (int i = 0; i < V; ++i)
printf("%d \t\t %d\n", i, dist[i]);
// Driver program to test methods of graph class
int main()
// create the graph given in above figure
int V = 7;
Graph g(V);
// making above shown graph
g.addEdge(0, 1, 2);
g.addEdge(0, 2, 6);
g.addEdge(1, 3, 5);
g.addEdge(2, 3, 8);
g.addEdge(4, 6, 2);
g.addEdge(5, 6, 6);
g.shortestPath(0);
return 0;
}
Final Answer:
Output
Complexity Analysis of Dijkstra Algorithm:

• Time complexity: O((V + E) log V), where V is the number of
vertices and E is the number of edges.
• Auxiliary Space: O(V), where V is the number of vertices and E
is the number of edges in the graph.
2. Array-based Implementation of Dijkstra’s Algorithm (Naive
Approach):
Dijkstra’s Algorithm can also be implemented using arrays without using
a priority queue. This implementation is straightforward but can be less
efficient, especially on large graphs.
The algorithm proceeds as follows:
• Initialize an array to store distances from the source to each
node.
• Mark all nodes as unvisited.
• Set the distance to the source node as 0 and infinity for all other
nodes.
• Repeat until all nodes are visited:
• Find the unvisited node with the smallest known
distance.
• For the current node, update the distances of its
unvisited neighbors.
• Mark the current node as visited.
Complexity Analysis:
• Time Complexity: O(V^2) in the worst case, where V is the
number of vertices. This can be improved to O(V^2) with some
optimizations.
• Auxiliary Space: O(V), where V is the number of vertices and E
is the number of edges in the graph.
Dijkstra’s Algorithms vs Bellman-Ford Algorithm
Dijkstra’s algorithm and Bellman-Ford algorithm are both used to find
the shortest path in a weighted graph, but they have some key
differences. Here are the main differences between Dijkstra’s algorithm
and Bellman-Ford algorithm:
Feature: Dijkstra’s Bellman Ford
Bellman-Ford
optimized for finding
algorithm is optimized
the shortest path
for finding the shortest
between a single
path between a single
source node and all
source node and all
other nodes in a graph
with non-negative
with negative edge
edge weights
Optimization weights.
Dijkstra’s algorithm the Bellman-Ford

uses a greedy algorithm relaxes all
approach where it edges in each iteration,
chooses the node with updating the distance
the smallest distance of each node by
and updates its considering all possible
Relaxation neighbors paths to that node
Dijkstra’s algorithm has Bellman-Ford

a time complexity of algorithm has a time
O(V^2) for a dense complexity of O(VE),
graph and O(E log V) where V is the number
Time Complexity
for a sparse graph, of vertices and E is the
Feature: Dijkstra’s Bellman Ford
where V is the number number of edges in the

of vertices and E is the graph.
number of edges in the
graph.
Dijkstra’s algorithm
Bellman-Ford
does not work with
algorithm can handle
graphs that have
negative edge weights
negative edge weights,
and can detect
as it assumes that all
negative-weight cycles
edge weights are non-
in the graph.
Negative Weights negative.
Dijkstra’s Algorithm vs Floyd-Warshall Algorithm

Dijkstra’s algorithm and Floyd-Warshall algorithm are both used to find
the shortest path in a weighted graph, but they have some key
differences. Here are the main differences between Dijkstra’s algorithm
and Floyd-Warshall algorithm:
Floyd-Warshall
Feature: Dijkstra’s Algorithm
Optimized for finding

the shortest path Floyd-Warshall
between a single algorithm is optimized
source node and all for finding the shortest
other nodes in a graph path between all pairs
with non-negative of nodes in a graph.
Optimization edge weights
Dijkstra’s algorithm is a Floyd-Warshall

single-source shortest algorithm, on the other
Technique
path algorithm that hand, is an all-pairs
Floyd-Warshall
Feature: Dijkstra’s Algorithm
uses a greedy shortest path algorithm

approach and that uses dynamic
calculates the shortest programming to
path from the source calculate the shortest
node to all other nodes path between all pairs
in the graph. of nodes in the graph.
Dijkstra’s algorithm Floyd-Warshall

has a time complexity algorithm, on the other
of O(V^2) for a dense hand, is an all-pairs
graph and O(E log V) shortest path algorithm
for a sparse graph, that uses dynamic
where V is the number programming to
of vertices and E is the calculate the shortest
number of edges in the path between all pairs
Time Complexity graph. of nodes in the graph.
Floyd-Warshall
Dijkstra’s algorithm algorithm, on the other
does not work with hand, is an all-pairs
graphs that have shortest path algorithm
negative edge weights, that uses dynamic
as it assumes that all programming to
edge weights are non- calculate the shortest
negative. path between all pairs
Negative Weights of nodes in the graph.
Dijkstra’s Algorithm vs A* Algorithm

Dijkstra’s algorithm and A* algorithm are both used to find the shortest
path in a weighted graph, but they have some key differences. Here are
the main differences between Dijkstra’s algorithm and A* algorithm:
Feature: A* Algorithm
Optimized for finding

A* algorithm is an
the shortest path
informed search
between a single
algorithm that uses a
source node and all
heuristic function to
guide the search
with non-negative
towards the goal node.
Search Technique edge weights
A* algorithm uses a
heuristic function that
estimates the distance
Dijkstra’s algorithm, from the current node
does not use any to the goal node. This
heuristic function and heuristic function is
considers all the nodes admissible, meaning
in the graph. that it never
overestimates the
actual distance to the
Heuristic Function goal node
Dijkstra’s algorithm has

a time complexity of
O(V^2) for a dense
The time complexity of
graph and O(E log V)
A* algorithm depends
for a sparse graph,
on the quality of the
where V is the number
heuristic function.
of vertices and E is the
number of edges in the
Time Complexity graph.
Dijkstra’s algorithm is . A* algorithm is

Application
used in many commonly used in
Feature: A* Algorithm
applications such as pathfinding and graph

routing algorithms, traversal problems,
GPS navigation such as video games,
systems, and network robotics, and planning
analysis algorithms.
Naive algorithm for Pattern Searching and

String Matching
Given text string with length n and a pattern with length m, the task is
to prints all occurrences of pattern in text.
Note: You may assume that n > m.
Examples:
Input: text = “THIS IS A TEST TEXT”, pattern = “TEST”
Output: Pattern found at index 10
Input: text = “AABAACAADAABAABA”, pattern = “AABA”
Output: Pattern found at index 0, Pattern found at index 9, Pattern found
at index 12
#include <iostream>
#include <string>
void search(string& pat, string& txt) {

int M = pat.size();
int N = txt.size();
// A loop to slide pat[] one by one

for (int i = 0; i <= N - M; i++) {
int j;
// For current index i, check for pattern match

for (j = 0; j < M; j++) {
if (txt[i + j] != pat[j]) {
break;
}
}
// If pattern matches at index i

if (j == M) {
cout << "Pattern found at index " << i << endl;
}
}
}
// Driver's Code
int main() {
// Example 1
string txt1 = "AABAACAADAABAABA";
string pat1 = "AABA";
cout << "Example 1: " << endl;
search(pat1, txt1);
// Example 2
string txt2 = "agd";
string pat2 = "g";
cout << "\nExample 2: " << endl;
search(pat2, txt2);
return 0;
}
Output
Pattern found at index 0
Time Complexity: O(N2)
Auxiliary Space: O(1)
Complexity Analysis of Naive algorithm for Pattern Searching:
Best Case: O(n)
• When the pattern is found at the very beginning of the text (or very
early on).
• The algorithm will perform a constant number of comparisons, typically
on the order of O(n) comparisons, where n is the length of the pattern.
Worst Case: O(n2)
• When the pattern doesn’t appear in the text at all or appears only at the
very end.
• The algorithm will perform O((n-m+1)*m) comparisons, where n is the
length of the text and m is the length of the pattern.
• In the worst case, for each position in the text, the algorithm may need
to compare the entire pattern against the text.
Rabin-Karp Algorithm for Pattern Searching
Given a text T[0. . .n-1] and a pattern P[0. . .m-1], write a function search(char P[],
char T[]) that prints all occurrences of P[] present in T[] using Rabin Karp
algorithm. You may assume that n > m.
Examples:
Input: T[] = “THIS IS A TEST TEXT”, P[] = “TEST”
Input: T[] = “AABAACAADAABAABA”, P[] = “AABA”
Rabin-Karp Algorithm:
In the Naive String Matching algorithm, we check whether every
substring of the text of the pattern’s size is equal to the pattern or not
one by one.
Like the Naive Algorithm, the Rabin-Karp algorithm also check every
substring. But unlike the Naive algorithm, the Rabin Karp algorithm
matches the hash value of the pattern with the hash value of the current
substring of text, and if the hash values match then only it starts
matching individual characters. So Rabin Karp algorithm needs to
calculate hash values for the following strings.
• Pattern itself
• All the substrings of the text of length m which is the size of
pattern.
How is Hash Value calculated in Rabin-Karp?
Hash value is used to efficiently check for potential matches between
a pattern and substrings of a larger text. The hash value is calculated
using a rolling hash function, which allows you to update the hash value
for a new substring by efficiently removing the contribution of the old
character and adding the contribution of the new character. This makes it
possible to slide the pattern over the text and calculate the hash value
for each substring without recalculating the entire hash from scratch.
Here’s how the hash value is typically calculated in Rabin-Karp:
Step 1: Choose a suitable base and a modulus:
• Select a prime number ‘p‘ as the modulus. This choice helps
avoid overflow issues and ensures a good distribution of hash
values.
• Choose a base ‘b‘ (usually a prime number as well), which is
often the size of the character set (e.g., 256 for ASCII
characters).
Step 2: Initialize the hash value:
• Set an initial hash value ‘hash‘ to 0.
Step 3: Calculate the initial hash value for the pattern:
• Iterate over each character in the pattern from left to right.
• For each character ‘c’ at position ‘i’, calculate its contribution to
the hash value as ‘c * (bpattern_length – i – 1) % p’ and add it to
‘hash‘.
• This gives you the hash value for the entire pattern.
Step 4: Slide the pattern over the text:
• Start by calculating the hash value for the first substring of
the text that is the same length as the pattern.
Step 5: Update the hash value for each subsequent substring:
• To slide the pattern one position to the right, you remove the
contribution of the leftmost character and add the contribution
of the new character on the right.
• The formula for updating the hash value when moving from
position ‘i’ to ‘i+1’ is:
hash = (hash - (text[i - pattern_length] * (bpattern_length - 1)) % p)
* b + text[i]
Step 6: Compare hash values:
• When the hash value of a substring in the text matches the
hash value of the pattern, it’s a potential match.
• If the hash values match, we should perform a character-by-
character comparison to confirm the match, as hash
collisions can occur.
Below is the Illustration of above algorithm:
Step-by-step approach:
• Initially calculate the hash value of the pattern.
• Start iterating from the starting of the string:
• Calculate the hash value of the current substring having
length m.
• If the hash value of the current substring and the
pattern are same check if the substring is same as the
pattern.
• If they are same, store the starting index as a valid
answer. Otherwise, continue for the next substrings.
• Return the starting indices as the required answer.
/* Following program is a C++ implementation of Rabin Karp
Algorithm given in the CLRS book */
// d is the number of characters in the input alphabet

#define d 256
/* pat -> pattern
txt -> text
q -> A prime number
*/
void search(char pat[], char txt[], int q)
int M = strlen(pat);
int N = strlen(txt);
int i, j;
int p = 0; // hash value for pattern
int t = 0; // hash value for txt
int h = 1;
// The value of h would be "pow(d, M-1)%q"
for (i = 0; i < M - 1; i++)
h = (h * d) % q;
// Calculate the hash value of pattern and first
// window of text
for (i = 0; i < M; i++) {
p = (d * p + pat[i]) % q;
t = (d * t + txt[i]) % q;
// Slide the pattern over text one by one
for (i = 0; i <= N - M; i++) {
// Check the hash values of current window of text

// and pattern. If the hash values match then only
// check for characters one by one
if (p == t) {
/* Check for characters one by one */
for (j = 0; j < M; j++) {
if (txt[i + j] != pat[j]) {
break;
// if p == t and pat[0...M-1] = txt[i, i+1,
// ...i+M-1]
if (j == M)
cout << "Pattern found at index " << i
<< endl;
// Calculate hash value for next window of text:
// Remove leading digit, add trailing digit
if (i < N - M) {
t = (d * (t - txt[i] * h) + txt[i + M]) % q;
// We might get negative value of t, converting
// it to positive
if (t < 0)
t = (t + q);
}
/* Driver code */
int main()
char txt[] = "GEEKS FOR GEEKS";
char pat[] = "GEEK";
// we mod to avoid overflowing of value but we should
// take as big q as possible to avoid the collison
int q = INT_MAX;
// Function Call
search(pat, txt, q);
return 0;
// This is code is contributed by rathbhupendra
Output
Time Complexity:
• The average and best-case running time of the Rabin-Karp
algorithm is O(n+m), but its worst-case time is O(nm).
• The worst case of the Rabin-Karp algorithm occurs when all
characters of pattern and text are the same as the hash values
of all the substrings of T[] match with the hash value of P[].
Auxiliary Space: O(1)
Limitations of Rabin-Karp Algorithm
Spurious Hit: When the hash value of the pattern matches with the hash
value of a window of the text but the window is not the actual pattern
then it is called a spurious hit. Spurious hit increases the time complexity
of the algorithm. In order to minimize spurious hit, we use good hash
function. It greatly reduces the spurious hit.
Finite Automata algorithm for Pattern
Searching
Given a text txt[0..n-1] and a pattern pat[0..m-1], write a
function search(char pat[], char txt[]) that prints all occurrences
of pat[] in txt[]. You may assume that n > m.
Examples:
Input: txt[] = "THIS IS A TEST TEXT"
pat[] = "TEST"
Input: txt[] = "AABAACAADAABAABA"

pat[] = "AABA"
Pattern searching is an important problem in computer science. When we

do search for a string in notepad/word file or browser or database,
pattern searching algorithms are used to show the search results.
The string-matching automaton is a very useful tool which is used in

string matching algorithm.
String matching algorithms build a finite automaton scans the text string
T for all occurrences of the pattern P.
FINITE AUTOMATA
• Idea of this approach is to build finite automata to scan text T for
finding all occurrences of pattern P.
• This approach examines each character of text exactly once to
find the pattern. Thus it takes linear time for matching but
preprocessing time may be large.
• It is defined by tuple M = {Q, ?, q, F, d} Where Q = Set of States
in finite automata
?=Sets of input symbols
q. = Initial state
F = Final State
? = Transition function
• Time Complexity = O(M³|?|)
A finite automaton M is a 5-tuple (Q, q0,A,??), where
Q is a finite set of states,
q0 ? Q is the start state,
A ? Q is a notable set of accepting states,
? is a finite input alphabet,
? is a function from Q x ? into Q called the transition function of M.
The finite automaton starts in state q0 and reads the characters of its
input string one at a time. If the automaton is in state q and reads input
character a, it moves from state q to state ? (q, a). Whenever its current
state q is a member of A, the machine M has accepted the string read so
far. An input that is not allowed is rejected.
A finite automaton M induces a function ? called the called the final-state
function, from ?* to Q such that ?(w) is the state M ends up in after
scanning the string w. Thus, M accepts a string w if and only if ?(w) ? A.
Algorithm-
FINITE AUTOMATA (T, P)
State <- 0
for l <- 1 to n
State <- ?(State, ti)
If State == m then
Match Found
end
end
Why it is efficient?
These string matching automaton are very efficient because they
examine each text character exactly once, taking constant time per text
character. The matching time used is O(n) where n is the length of Text
string.
But the preprocessing time i.e. the time taken to build the finite
automaton can be large if ? is large.
Before we discuss Finite Automaton construction, let us take a look at
the following Finite Automaton for pattern ACACAGA.
The above diagrams represent graphical and tabular representations of
pattern ACACAGA.
Number of states in Finite Automaton will be M+1 where M is length of
the pattern. The main thing to construct Finite Automaton is to get the
next state from the current state for every possible character.
Given a character x and a state k, we can get the next state by
considering the string “pat[0..k-1]x” which is basically concatenation of
pattern characters pat[0], pat[1] …pat[k-1] and the character x. The idea
is to get length of the longest prefix of the given pattern such that the
prefix is also suffix of “pat[0..k-1]x”. The value of length gives us the next
state.
For example, let us see how to get the next state from current state 5
and character ‘C’ in the above diagram. We need to consider the string,
“pat[0..4]C” which is “ACACAC”. The length of the longest prefix of the
pattern such that the prefix is suffix of “ACACAC”is 4 (“ACAC”). So the
next state (from state 5) is 4 for character ‘C’.
In the following code, computeTF() constructs the Finite Automaton. The
time complexity of the computeTF() is O(m^3*NO_OF_CHARS) where m
is length of the pattern and NO_OF_CHARS is size of alphabet (total
number of possible characters in pattern and text). The implementation
tries all possible prefixes starting from the longest possible that can be a
suffix of “pat[0..k-1]x”. There are better implementations to construct
Finite Automaton in O(m*NO_OF_CHARS) (Hint: we can use something
like lps array construction in KMP algorithm).
// CPP program for Finite Automata Pattern searching
// Algorithm
#define NO_OF_CHARS 256
int getNextState(string pat, int M, int state, int x)

{
// If the character c is same as next character
// in pattern,then simply increment state
if (state < M && x == pat[state])
return state+1;
// ns stores the result which is next state

int ns, i;
// ns finally contains the longest prefix

// which is also suffix in "pat[0..state-1]c"
// Start from the largest possible value

// and stop when you find a prefix which
// is also suffix
for (ns = state; ns > 0; ns--)
{
if (pat[ns-1] == x)
{
for (i = 0; i < ns-1; i++)
if (pat[i] != pat[state-ns+1+i])
break;
if (i == ns-1)
return ns;
}
}
return 0;
}
/* This function builds the TF table which represents4

Finite Automata for a given pattern */
void computeTF(string pat, int M, int TF[][NO_OF_CHARS])
{
int state, x;
for (state = 0; state <= M; ++state)
for (x = 0; x < NO_OF_CHARS; ++x)
TF[state][x] = getNextState(pat, M, state, x);
}
/* Prints all occurrences of pat in txt */

void search(string pat, string txt)
{
int M = pat.size();
int N = txt.size();
int TF[M+1][NO_OF_CHARS];
computeTF(pat, M, TF);
// Process txt over FA.

int i, state=0;
for (i = 0; i < N; i++)
{
state = TF[state][txt[i]];
if (state == M)
cout<<" Pattern found at index "<< i-M+1<<endl;
}
}
// Driver program to test above function

int main()
{
string txt = "AABAACAADAABAAABAA";
string pat = "AABA";
search(pat, txt);
return 0;
}
Output:

Time Complexity: O(m2)

Auxiliary Space: O(m)
KMP Algorithm for Pattern Searching
Given a text txt[0 . . . N-1] and a pattern pat[0 . . . M-1], write a function
search(char pat[], char txt[]) that prints all occurrences of pat[] in txt[].
You may assume that N > M.
Examples:
Input: txt[] = “THIS IS A TEST TEXT”, pat[] = “TEST”
Input: txt[] = “AABAACAADAABAABA”
pat[] = “AABA”
Output: Pattern found at index 0, Pattern found at index 9, Pattern found
at index 12
Arrivals of pattern in the text
We have discussed the Naive pattern-searching algorithm in

the previous post. The worst case complexity of the Naive algorithm is
O(m(n-m+1)). The time complexity of the KMP algorithm is O(n+m) in the
worst case.
KMP (Knuth Morris Pratt) Pattern Searching:
The Naive pattern-searching algorithm doesn’t work well in cases where
we see many matching characters followed by a mismatching character.
Examples:
1) txt[] = “AAAAAAAAAAAAAAAAAB”, pat[] = “AAAAB”
2) txt[] = “ABABABCABABABCABABABC”, pat[] = “ABABAC” (not a
worst case, but a bad case for Naive)
The KMP matching algorithm uses degenerating property (pattern having
the same sub-patterns appearing more than once in the pattern) of the
pattern and improves the worst-case complexity to O(n+m).
The basic idea behind KMP’s algorithm is: whenever we detect a
mismatch (after some matches), we already know some of the characters
in the text of the next window. We take advantage of this information to
avoid matching the characters that we know will anyway match.
Matching Overview
txt = “AAAAABAAABA”
pat = “AAAA”
We compare first window of txt with pat
pat = “AAAA” [Initial position]
We find a match. This is same as Naive String Matching.
In the next step, we compare next window of txt with pat.
pat = “AAAA” [Pattern shifted one position]
This is where KMP does optimization over Naive. In this second window,
we only compare fourth A of pattern
with fourth character of current window of text to decide whether current
window matches or not. Since we know
first three characters will anyway match, we skipped matching first three
characters.
Need of Preprocessing?
An important question arises from the above explanation, how to know
how many characters to be skipped. To know this,
we pre-process pattern and prepare an integer array lps[] that tells us
the count of characters to be skipped
Preprocessing Overview:
• KMP algorithm preprocesses pat[] and constructs an
auxiliary lps[] of size m (same as the size of the pattern) which
is used to skip characters while matching.
• Name lps indicates the longest proper prefix which is also a
suffix. A proper prefix is a prefix with a whole string not
allowed. For example, prefixes of “ABC” are “”, “A”, “AB” and
“ABC”. Proper prefixes are “”, “A” and “AB”. Suffixes of the string
are “”, “C”, “BC”, and “ABC”.
• We search for lps in subpatterns. More clearly we focus on sub-
strings of patterns that are both prefix and suffix.
• For each sub-pattern pat[0..i] where i = 0 to m-1, lps[i] stores
the length of the maximum matching proper prefix which is also
a suffix of the sub-pattern pat[0..i].
lps[i] = the longest proper prefix of pat[0..i] which is also a suffix of
pat[0..i].
Note: lps[i] could also be defined as the longest prefix which is also a
proper suffix. We need to use it properly in one place to make sure that
the whole substring is not considered.
Examples of lps[] construction:
For the pattern “AAAA”, lps[] is [0, 1, 2, 3]
For the pattern “ABCDE”, lps[] is [0, 0, 0, 0, 0]
For the pattern “AABAACAABAA”, lps[] is [0, 1, 0, 1, 2, 0, 1, 2, 3, 4, 5]
For the pattern “AAACAAAAAC”, lps[] is [0, 1, 2, 0, 1, 2, 3, 3, 3, 4]
For the pattern “AAABAAA”, lps[] is [0, 1, 2, 0, 1, 2, 3]
Preprocessing Algorithm:
In the preprocessing part,
• We calculate values in lps[]. To do that, we keep track of the
length of the longest prefix suffix value (we use len variable for
this purpose) for the previous index
• We initialize lps[0] and len as 0.
• If pat[len] and pat[i] match, we increment len by 1 and assign
the incremented value to lps[i].
• If pat[i] and pat[len] do not match and len is not 0, we update
len to lps[len-1]
• See computeLPSArray() in the below code for details
Illustration of preprocessing (or construction of lps[]):
pat[] = “AAACAAAA”
=> len = 0, i = 0:
• lps[0] is always 0, we move to i = 1
=> len = 0, i = 1:
• Since pat[len] and pat[i] match, do len++,
• store it in lps[i] and do i++.
• Set len = 1, lps[1] = 1, i = 2
=> len = 1, i = 2:
• store it in lps[i] and do i++.
• Set len = 2, lps[2] = 2, i = 3
=> len = 2, i = 3:
• Since pat[len] and pat[i] do not match, and len > 0,
• Set len = lps[len-1] = lps[1] = 1
=> len = 1, i = 3:
• Since pat[len] and pat[i] do not match and len > 0,
• len = lps[len-1] = lps[0] = 0
=> len = 0, i = 3:
• Since pat[len] and pat[i] do not match and len = 0,
• Set lps[3] = 0 and i = 4
=> len = 0, i = 4:
• Store it in lps[i] and do i++.
• Set len = 1, lps[4] = 1, i = 5
=> len = 1, i = 5:
• Set len = 2, lps[5] = 2, i = 6
=> len = 2, i = 6:
• len = 3, lps[6] = 3, i = 7
=> len = 3, i = 7:
• Since pat[len] and pat[i] do not match and len > 0,
• Set len = lps[len-1] = lps[2] = 2
=> len = 2, i = 7:
• len = 3, lps[7] = 3, i = 8
We stop here as we have constructed the whole lps[].
Implementation of KMP algorithm:
Unlike the Naive algorithm, where we slide the pattern by one and
compare all characters at each shift, we use a value from lps[] to decide
the next characters to be matched. The idea is to not match a character
that we know will anyway match.
How to use lps[] to decide the next positions (or to know the number of
characters to be skipped)?
• We start the comparison of pat[j] with j = 0 with characters of
the current window of text.
• We keep matching characters txt[i] and pat[j] and keep
incrementing i and j while pat[j] and txt[i] keep matching.
• When we see a mismatch
• We know that characters pat[0..j-1] match with txt[i-
j…i-1] (Note that j starts with 0 and increments it only
when there is a match).
• We also know (from the above definition) that lps[j-1]
is the count of characters of pat[0…j-1] that are both
proper prefix and suffix.
• From the above two points, we can conclude that we do
not need to match these lps[j-1] characters with txt[i-
j…i-1] because we know that these characters will
anyway match. Let us consider the above example to
understand this.
Below is the illustration of the above algorithm:
Consider txt[] = “AAAAABAAABA“, pat[] = “AAAA“
If we follow the above LPS building process then lps[] = {0, 1, 2, 3}
-> i = 0, j = 0: txt[i] and pat[j] match, do i++, j++
-> i = 4, j = 4: Since j = M, print pattern found and reset j, j = lps[j-1] =
lps[3] = 3
Here unlike Naive algorithm, we do not match first three
characters of this window. Value of lps[j-1] (in above step) gave us index
of next character to match.
-> i = 5, j = 4: Since j == M, print pattern found and reset j, j = lps[j-1] =
lps[3] = 3
Again unlike Naive algorithm, we do not match first three characters of
this window. Value of lps[j-1] (in above step) gave us index of next
character to match.
-> i = 5, j = 3: txt[i] and pat[j] do NOT match and j > 0, change only j. j =
lps[j-1] = lps[2] = 2
lps[j-1] = lps[1] = 1
lps[j-1] = lps[0] = 0
-> i = 5, j = 0: txt[i] and pat[j] do NOT match and j is 0, we do i++.
-> i = 6, j = 0: txt[i] and pat[j] match, do i++ and j++
-> i = 7, j = 1: txt[i] and pat[j] match, do i++ and j++
// C++ program for implementation of KMP pattern searching

// algorithm
void computeLPSArray(char* pat, int M, int* lps);
// Prints occurrences of pat[] in txt[]

void KMPSearch(char* pat, char* txt)
{
int M = strlen(pat);
int N = strlen(txt);
// create lps[] that will hold the longest prefix suffix

// values for pattern
int lps[M];
// Preprocess the pattern (calculate lps[] array)

computeLPSArray(pat, M, lps);
int i = 0; // index for txt[]

int j = 0; // index for pat[]
while ((N - i) >= (M - j)) {
if (pat[j] == txt[i]) {
j++;
i++;
}
if (j == M) {
printf("Found pattern at index %d ", i - j);
j = lps[j - 1];
}
// mismatch after j matches

else if (i < N && pat[j] != txt[i]) {
// Do not match lps[0..lps[j-1]] characters,
// they will match anyway
if (j != 0)
j = lps[j - 1];
else
i = i + 1;
}
}
}
// Fills lps[] for given pattern pat[0..M-1]

void computeLPSArray(char* pat, int M, int* lps)
{
// length of the previous longest prefix suffix
int len = 0;
lps[0] = 0; // lps[0] is always 0
// the loop calculates lps[i] for i = 1 to M-1

int i = 1;
while (i < M) {
if (pat[i] == pat[len]) {
len++;
lps[i] = len;
i++;
}
else // (pat[i] != pat[len])
{
// This is tricky. Consider the example.
// AAACAAAA and i = 7. The idea is similar
// to search step.
if (len != 0) {
len = lps[len - 1];
// Also, note that we do not increment

// i here
}
else // if (len == 0)
{
lps[i] = 0;
i++;
}
}
}
}
// Driver code
int main()
{
char txt[] = "ABABDABACDABABCABAB";
char pat[] = "ABABCABAB";
KMPSearch(pat, txt);
return 0;
}
Output
Found pattern at index 10
Time Complexity: O(N+M) where N is the length of the text and M is the
length of the pattern to be found.
Auxiliary Space: O(M)

Greedy Algorithms

Uploaded by

Copyright:

Available Formats

Greedy Algorithms

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Greedy Algorithms

Uploaded by

Copyright:

Available Formats

Greedy Algorithms

Greedy algorithms are a class of algorithms that make locally

What is Greedy Algorithm?

The Matroid Greedy Algorithm is a method for finding an optimal

Here's a high-level overview of the Matroid Greedy Algorithm:

1. Definition of a Matroid: A matroid is defined by a finite ground

Overall, the Matroid Greedy Algorithm offers an efficient and elegant

Activity Selection Problem

// C++ program for activity selection problem.

// The following implementation assumes that the activities

// Prints a maximum set of activities that can be done by a

// Consider rest of the activities

Fractional Knapsack Problem

// C++ program to solve fractional Knapsack Problem

// Structure for an item which stores weight and

// Comparison function to sort Item

// Main greedy function to solve problem

double finalvalue = 0.0;

// Looping through all items

// If adding Item won't overflow,

// If we can't add current Item,

The method which is used to construct optimal prefix code is

temp <- get node ()

left (temp] Get_min (Q) right [temp] Get Min (Q)

a = left [templ b = right [temp]

F [temp]<- f[a] + [b]

insert (Q, temp)

return Get_min (0)

Steps to build Huffman Tree

3. Create a new internal node with a frequency equal to the sum of

Now min heap contains 3 nodes.

Now min heap contains 2 nodes.

Now min heap contains only one node.

The codes are as follows:

// A Huffman tree node

// One of the input characters

// Frequency of the character

// Left and right child of this node

// A Min Heap: Collection of

// Current size of min heap

// capacity of min heap

// Array of minheap node pointers

// A utility function allocate a new

temp->left = temp->right = NULL;

// A utility function to create

struct MinHeap* minHeap

minHeap->array = (struct MinHeapNode**)malloc(

struct MinHeapNode* t = *a;

// The standard minHeapify function.

int smallest = idx;

if (left < minHeap->size

if (right < minHeap->size

// A utility function to check

return (minHeap->size == 1);

// A standard function to extract

struct MinHeapNode* temp = minHeap->array[0];

// A utility function to insert

minHeap->array[i] = minHeap->array[(i - 1) / 2];

// A standard function to build min heap

for (i = (n - 1) / 2; i >= 0; --i)