UNIT 3

Introduction to Minimum Spanning Tree:

The minimum spanning tree algorithm is developed by referencing the field of graph theory in
mathematics. Thus, to understand this algorithm, we shall first understand what a spanning tree
is?

A spanning tree of a connected undirected graph is a subgraph, i.e., a tree structure that binds all
vertices with a minimum edge cost sum. If you have graph G with vertices V and edges E, then
that graph can be represented as G(V, E). For this graph G(V, E), if you construct a tree structure
G’(V’, E’) such that the formed tree structure follows constraints mentioned below, then that
structure can be called a Spanning Tree.

1. V’ = V (number of Vertices in G’ must be equal to the number of vertices in G)

2. E’ = |V| - 1 (Edges of G’ must be equal to the number of vertices in graph G minus 1)

Let's create spanning trees for a given graph topology to understand this concept better.

For the graph above, possible spanning tree structures are:

Now, let’s look at the sum of edge weight costs for all these spanning trees represented in the
table below:

Spanning Tree Sum of Edge Costs

ST - 1 22

ST – 2 35
 ST – 3 36

Table - 1: Sum of Edge costs

The spanning tree structure 1 has an overall edge weight cost of 22. Which is the least possible
substructure cost for given graph G. Thus, ST-1 is considered the minimum spanning tree of
graph G. A minimum spanning tree in a weighted graph is one that has the least weight of all the
other spanning tree structures.

HOW to Find the Minimum Spanning Tree?

The naive method we discussed above is not an ideal approach to find out possible MST
structures. In that method, we will have to generate all possible spanning trees at first and then
calculate the overall sum of edge weights for each generated spanning tree. After that, we will
have to determine the minima of all those spanning trees, which will cost us more time and
memory.
A better method is to locate a vital attribute of the MST, which will provide clarification if some
particular edge should be included in it or not. And then, we can use that property to build up the
MST gradually. Let’s find out how we can do that with the help of the greedy programming
paradigm. The greedy algorithms that we can use are Prim’s Algorithm and Kruskal’s
Algorithm.
First, we shall look into Prim’s algorithm.
1. Prim’s Algorithm
Prim's algorithm begins with a single node and adds up adjacent nodes one by one by
discovering all of the connected edges along the way. Edges with the lowest weights that don't
generate cycles are chosen for inclusion in the MST structure. As a result, we can claim that
Prim's algorithm finds the globally best answer by making locally optimal decisions.
Steps involved in Prim’s algorithms are mentioned below:
Step 1: Choose any vertex as a starting vertex.

 Step 2: Pick an edge connecting any tree vertex and fringe vertex (adjacent vertex to visited
vertex) having the minimum edge weight.

 Step 3: Add the selected edge to MST only if it doesn't form any closed cycle.

 Step 4: Keep repeating steps 2 and 3 until the fringe vertices exist.

 Step 5: End.
 Kruskal’s Algorithm
Kruskal's approach sorts all the edges in ascending order of edge weights and only adds nodes to
the tree if the chosen edge does not form a cycle. It also selects the edge with the lowest cost first
and the edge with the highest cost last. As a result, we can say that the Kruskal algorithm makes
a locally optimum decision in the hopes of finding the global optimal solution. Hence, this
algorithm can also be considered as a Greedy Algorithm.
The steps involved in Kruskal’s algorithm to generate a minimum spanning tree are:
Step 1: Sort all edges in increasing order of their edge weights.
Step 2: Pick the smallest edge.
Step 3: Check if the new edge creates a cycle or loop in a spanning tree.
Step 4: If it doesn’t form the cycle, then include that edge in MST. Otherwise, discard it.
Step 5: Repeat from step 2 until it includes |V| - 1 edges in MST.
Applications of Minimum Spanning Tree
The following are the applications of minimum spanning tree in data structures:
Telecommunication Network Building: A basic naive approach will be more expensive if we
develop a telecommunication network for the entire city. Using the MST approach in data
structures, we can design a communication system at a much lesser cost. The distinction between
Naive and MST routing is illustrated in the diagram below.

 Constructing Highways or Railroads: The Minimum Spanning Tree (MST) technique is used
globally for building roadways or railroads. The MST approach determines the best route
between two cities depending on all potential routes. Essentially, the algorithm treats cities as
 vertices and roads connecting them as edges to produce a subtree that connects two cities with
less cost.

Bellman-Ford Algorithm
Bellman-Ford is a single source shortest path algorithm that determines the shortest path between
a given source vertex and every other vertex in a graph. This algorithm can be used on both
weighted and unweighted graphs.
A Bellman-Ford algorithm is also guaranteed to find the shortest path in a graph, similar
to Dijkstra’s algorithm . Although Bellman-Ford is slower than Dijkstra’s algorithm, it is capable
of handling graphs with negative edge weights, which makes it more versatile. The shortest
path cannot be found if there exists a negative cycle in the graph. If we continue to go around the
negative cycle an infinite number of times, then the cost of the path will continue to decrease
(even though the length of the path is increasing). As a result, Bellman-Ford is also capable of
detecting negative cycles, which is an important feature.
Why would one ever have edges with negative weights in real life?
Negative weight edges might seem useless at first but they can explain a lot of phenomena like
cashflow, the heat released/absorbed in a chemical reaction, etc.
For instance, if there are different ways to reach from one chemical A to another chemical B,
each method will have sub-reactions involving both heat dissipation and absorption.
If we want to find the set of reactions where minimum energy is required, then we will need to
be able to factor in the heat absorption as negative weights and heat dissipation as positive
weights.

Why do we need to be careful with negative weights?

Negative weight edges can create negative weight cycles i.e. a cycle that will reduce the total
path distance by coming back to the same point.

Shortest path algorithms like Dijkstra's Algorithm that aren't able to detect such a cycle can give
an incorrect result because they can go through a negative weight cycle and reduce the path
length.
How Bellman Ford's algorithm works
Bellman Ford algorithm works by overestimating the length of the path from the starting vertex
to all other vertices. Then it iteratively relaxes those estimates by finding new paths that are
shorter than the previously overestimated paths.
Single Source Shortest Path in a directed Acyclic Graphs

topological sort of its vertices, we can figure out shortest paths from a single source in ∅(V+E)
By relaxing the edges of a weighted DAG (Directed Acyclic Graph) G = (V, E) according to a

time. Shortest paths are always well described in a dag, since even if there are negative-weight
edges, no negative-weight cycles can exist.
DAG - SHORTEST - PATHS (G, w, s)
1. Topologically sort the vertices of G.
2. INITIALIZE - SINGLE- SOURCE (G, s)
3. for each vertex u taken in topologically sorted order
4. do for each vertex v ∈ Adj [u]
5. do RELAX (u, v, w)

topological sort can be implemented in ∅ (V + E) time. In the for loop of lines 3 - 5, as in

The running time of this data is determined by line 1 and by the for loop of lines 3 - 5. The

Dijkstra's algorithm, there is one repetition per vertex. For each vertex, the edges that leave the

edge. The running time is thus ∅ (V + E), which is linear in the size of an adjacency list
vertex are each examined exactly once. Unlike Dijkstra's algorithm, we use only O (1) time per

depiction of the graph.

Example:
Step1: To topologically sort vertices apply DFS (Depth First Search) and then arrange vertices
in linear order by decreasing order of finish time.
Backward Skip 10sPlay VideoForward Skip 10s

Now, take each vertex in topologically sorted order and relax each edge.

adj [s] →t, x

0+3<∞
d [t] ← 3
0+2<∞
d [x] ← 2

adj [t] → r, x
3+1<∞
d [r] ← 4
3+5≤2

adj [x] → y
2-3<∞
d [y] ← -1

adj [y] → r
-1 + 4 < 4
3 <4
d [r] ← 3
 Thus the Shortest Path is:

1. s to x is 2
2. s to y is -1
3. s to t is 3
4. s to r is 3 .

An Introduction to Dijkstra's Algorithm

Now that we know some basic Graphs concepts let's dive into understanding the concept of
Dijkstra's Algorithm.
Ever wondered how does Google Maps finds the shortest and fastest route between two places?
Well, the answer is Dijkstra's Algorithm. Dijkstra's Algorithm is a Graph algorithm that
finds the shortest path from a source vertex to all other vertices in the Graph (single source
shortest path). It is a type of Greedy Algorithm that only works on Weighted Graphs having
positive weights. The time complexity of Dijkstra's Algorithm is O(V2) with the help of the
adjacency matrix representation of the graph. This time complexity can be reduced to O((V + E)
log V) with the help of an adjacency list representation of the graph, where V is the number of
vertices and E is the number of edges in the graph.
Fundamentals of Dijkstra's Algorithm
The following are the basic concepts of Dijkstra's Algorithm:
Dijkstra's Algorithm begins at the node we select (the source node), and it examines the graph to
find the shortest path between that node and all the other nodes in the graph.
The Algorithm keeps records of the presently acknowledged shortest distance from each node to
the source node, and it updates these values if it finds any shorter path.
Once the Algorithm has retrieved the shortest path between the source and another node, that
node is marked as 'visited' and included in the path.
The procedure continues until all the nodes in the graph have been included in the path. In this
manner, we have a path connecting the source node to all other nodes, following the shortest
possible path to reach each node.
Fundamentals of Dijkstra's Algorithm
The following are the basic concepts of Dijkstra's Algorithm:
Dijkstra's Algorithm begins at the node we select (the source node), and it examines the graph to
find the shortest path between that node and all the other nodes in the graph.
The Algorithm keeps records of the presently acknowledged shortest distance from each node to
the source node, and it updates these values if it finds any shorter path.
Once the Algorithm has retrieved the shortest path between the source and another node, that
node is marked as 'visited' and included in the path.
The procedure continues until all the nodes in the graph have been included in the path. In this
manner, we have a path connecting the source node to all other nodes, following the shortest
possible path to reach each node.
Understanding Dijkstra's Algorithm with an Example
The following is the step that we will follow to implement Dijkstra's Algorithm:
Step 1: First, we will mark the source node with a current distance of 0 and set the rest of the
nodes to INFINITY.
Step 2: We will then set the unvisited node with the smallest current distance as the current
node, suppose X.
Step 3: For each neighbor N of the current node X: We will then add the current distance of X
with the weight of the edge joining X-N. If it is smaller than the current distance of N, set it as
the new current distance of N.
Step 4: We will then mark the current node X as visited.
Step 5: We will repeat the process from 'Step 2' if there is any node unvisited left in the graph.
Let us now understand the implementation of the algorithm with the help of an example:
Figure 6: The Given Graph
1. We will use the above graph as the input, with node A as the source.
2. First, we will mark all the nodes as unvisited.
3. We will set the path to 0 at node A and INFINITY for all the other nodes.
4. We will now mark source node A as visited and access its neighboring nodes.
Note: We have only accessed the neighboring nodes, not visited them.
5. We will now update the path to node B by 4 with the help of relaxation because the path
to node A is 0 and the path from node A to B is 4, and the minimum((0 + 4),
INFINITY) is 4.
6. We will also update the path to node C by 5 with the help of relaxation because the path
to node A is 0 and the path from node A to C is 5, and the minimum((0 + 5),
INFINITY) is 5. Both the neighbors of node A are now relaxed; therefore, we can move
ahead.
7. We will now select the next unvisited node with the least path and visit it. Hence, we will
visit node B and perform relaxation on its unvisited neighbors. After performing
relaxation, the path to node C will remain 5, whereas the path to node E will become 11,
and the path to node D will become 13.
8. We will now visit node E and perform relaxation on its neighboring nodes B, D, and F.
Since only node F is unvisited, it will be relaxed. Thus, the path to node B will remain as
it is, i.e., 4, the path to node D will also remain 13, and the path to node F will become 14
(8 + 6).
9. Now we will visit node D, and only node F will be relaxed. However, the path to
node F will remain unchanged, i.e., 14.
10. Since only node F is remaining, we will visit it but not perform any relaxation as all its
neighboring nodes are already visited.
11. Once all the nodes of the graphs are visited, the program will end.

Hence, the final paths we concluded are:

A=0
B = 4 (A -> B)
C = 5 (A -> C)
D = 4 + 9 = 13 (A -> B -> D)
E = 5 + 3 = 8 (A -> C -> E)
F = 5 + 3 + 6 = 14 (A -> C -> E -> F)
Divide and Conquer Introduction
Divide and Conquer is an algorithmic pattern. In algorithmic methods, the design is to take a
dispute on a huge input, break the input into minor pieces, decide the problem on each of the
small pieces, and then merge the piecewise solutions into a global solution. This mechanism of
solving the problem is called the Divide & Conquer Strategy.
Divide and Conquer algorithm consists of a dispute using the following three steps.
Divide the original problem into a set of subproblems.
Conquer: Solve every subproblem individually, recursively.
Combine: Put together the solutions of the subproblems to get the solution to the whole
problem.

Generally, we can follow the divide-and-conquer approach in a three-step process.

Examples: The specific computer algorithms are based on the Divide & Conquer approach:
1. Maximum and Minimum Problem
2. Binary Search
3. Sorting (merge sort, quick sort)
4. Tower of Hanoi.
Max - Min Problem

Problem: Analyze the algorithm to find the maximum and minimum element from an array.

Algorithm: Max ?Min Element (a [])

Max: a [i]
Min: a [i]
For i= 2 to n do
If a[i]> max then
max = a[i]
if a[i] < min then
min: a[i]
return (max, min)
Analysis:

Method 1: if we apply the general approach to the array of size n, the number of comparisons
required are 2n-2.

Method-2: In another approach, we will divide the problem into sub-problems and find the max
and min of each group, now max. Of each group will compare with the only max of another
group and min with min.

Let n = is the size of items in an array

Let T (n) = time required to apply the algorithm on an array of size n. Here we divide the terms
as T(n/2).

2 here tends to the comparison of the minimum with minimum and maximum with maximum as
in above example.

T (n) = 2 T → Eq (i)

T (2) = 1, time required to compare two elements/items. (Time is measured in units of the
number of comparisons)

→ Eq (ii)
Put eq (ii) in eq (i)

Similarly, apply the same procedure recursively on each subproblem or anatomy

{Use recursion means, we will use some stopping condition to stop the algorithm}

Recursion will stop, when → (Eq. 4)

Put the equ.4 into equation3.

Number of comparisons requires applying the divide and conquering algorithm on n

elements/items =

Number of comparisons requires applying general approach on n elements = (n-1) + (n-1) = 2n-2

From this example, we can analyze, that how to reduce the number of comparisons by using this
technique.
Binary Search
1. In Binary Search technique, we search an element in a sorted array by recursively dividing the
interval in half.

2. Firstly, we take the whole array as an interval.

3. If the Pivot Element (the item to be searched) is less than the item in the middle of the interval,
We discard the second half of the list and recursively repeat the process for the first half of the
list by calculating the new middle and last element.

4. If the Pivot Element (the item to be searched) is greater than the item in the middle of the
interval, we discard the first half of the list and work recursively on the second half by
calculating the new beginning and middle element.

5. Repeatedly, check until the value is found or interval is empty.

Analysis:
Input: an array A of size n, already sorted in the ascending or descending order.
Output: analyze to search an element item in the sorted array of size n.
Logic: Let T (n) = number of comparisons of an item with n elements in a sorted array.

o Set BEG = 1 and END = n

o Find mid =
o Compare the search item with the mid item.

Case 1: item = A[mid], then LOC = mid, but it the best case and T (n) = 1

Case 2: item ≠A [mid], then we will split the array into two equal parts of size .

And again find the midpoint of the half-sorted array and compare with search element.

Repeat the same process until a search element is found.

T (n) = ...... (Equation 1)

{Time to compare the search element with mid element, then with half of the selected half part
of array}
At least there will be only one term left that's why that term will compare

out, and only one comparison be done that's why

Is the last term of the equation and it will be equal to 1

Quick Sort
Introduction
Quick Sort is one of the most popular and efficient sorting algorithms. It is generally the default
sorting algorithms in many programming languages (including C++ and Java).
Even though the worst case time complexity of Quick Sort is O(n^2), it works at O(n log n) in
most cases and is generally much faster than merge sort if implemented properly.
Quicksort is a divide-and-conquer algorithm. It works by selecting a 'pivot' element from the
array and partitioning the other elements into two sub-arrays, according to whether they are less
than or greater than the pivot. The sub-arrays are then sorted recursively.
Algorithm
Pick an element, called a pivot, from the array.
Partitioning: reorder the array so that all elements with values less than the pivot come before the
pivot, while all elements with values greater than the pivot come after it (equal values can go
either way). After this partitioning, the pivot is in its final position. This is called the partition
operation.
Recursively apply the above steps to the sub-array of elements with smaller values and
separately to the sub-array of elements with greater values.
The base case of the recursion is arrays of size zero or one, which are in order by definition, so
they never need to be sorted.
There are multiple variants of quick sort depending on the choice of pivot. Some popular choices
of pivots being:
First element of the unsorted array
Last element of the unsorted array
Middle element of the unsorted array
Random element from the unsorted array
The choice of pivot determines the chances of the algorithm hitting the worst case for the given
array.
Example

int partition (int arr[], int low, int high) {

int pivot = arr[high];
 int i = low - 1;

for (int j = low; j < high; j++) {

if (arr[j] < pivot) {
i++;
swap(&arr[i], &arr[j]);
}
}

swap(&arr[i + 1], &arr[high]);

return i + 1;
}

void quickSortUtil(int arr[], int low, int high) {

if (low >= high) {
return;
}

int pivot = partition (arr, low, high);

quickSortUtil(arr, low, pivot - 1);

quickSortUtil(arr, pivot + 1, high);
}

void quickSort(int arr[], int n) {

quickSortUtil(arr, 0, n - 1);
}

Merge Sort
Introduction
Merge Sort is also a Divide and Conquer algorithm similar to Quick Sort. The input array right
into two parts in the middle. The two haves are then sorted recursively and then merged to create
the sorted array.
Algorithm
The algorithm looks something like this:
Sort the left half of the array
Sort the right half of the array
Merge both the halves of the array
The base case of the recursion is arrays of size zero or one, which are in order by definition, so
they never need to be sorted.
Example

Strassen’s Matrix Multiplication

Strassen’s Matrix Multiplication is the divide and conquer approach to solve the matrix
multiplication problems. The usual matrix multiplication method multiplies each row with each
column to achieve the product matrix. The time complexity taken by this approach is O(n3),
since it takes two loops to multiply. Strassen’s method was introduced to reduce the time
complexity from O(n3) to O(nlog 7)
Procedure of Strassen matrix multiplication
There are some procedures:
Divide a matrix of order of 2*2 recursively till we get the matrix of 2*2.
Use the previous set of formulas to carry out 2*2 matrix multiplication.
In this eight multiplication and four additions, subtraction are performed.
Combine the result of two matrixes to find the final product or final matrix.
Formulas for Stassen's matrix multiplication
In Strassen's matrix multiplication there are seven multiplication and four addition, subtraction in
total.
1. D1 = (a11 + a22) (b11 + b22)
2. D2 = (a21 + a22).b11
3. D3 = (b12 – b22).a11
4. D4 = (b21 – b11).a22
5. D5 = (a11 + a12).b22
6. D6 = (a21 – a11) . (b11 + b12)
7. D7 = (a12 – a22) . (b21 + b22)

C11 = d1 + d4 – d5 + d7
C12 = d3 + d5
C21 = d2 + d4
C22 = d1 + d3 – d2 – d6
Algorithm for Strassen's matrix multiplication
Algorithm Strassen(n, a, b, d)
begin
If n = threshold then compute
C = a * b is a conventional matrix.
Else
 Partition a into four sub matrices a11, a12, a21, a22.
Partition b into four sub matrices b11, b12, b21, b22.
Strassen ( n/2, a11 + a22, b11 + b22, d1)
Strassen ( n/2, a21 + a22, b11, d2)
Strassen ( n/2, a11, b12 – b22, d3)
Strassen ( n/2, a22, b21 – b11, d4)
Strassen ( n/2, a11 + a12, b22, d5)
Strassen (n/2, a21 – a11, b11 + b22, d6)
Strassen (n/2, a12 – a22, b21 + b22, d7)

C = d1+d4-d5+d7 d3+d5
d2+d4 d1+d3-d2-d6

end if

return (C)
end.