UNIT 1

UNIT I

ROLE OF ALGORITHMS IN COMPUTING & COMPLEXITY ANALYSIS

Algorithms – Algorithms as a Technology -Time and Space complexity of algorithms- Asymptotic analysis-
Average and worst-case analysis-Asymptotic notation-Importance of efficient algorithms- Program
performance measurement - Recurrences: The Substitution Method – The Recursion-Tree Method- Data
structures and algorithms.

Program Performance Measurement

In computer science, there are numerous algorithms

for solving a problem. When there are several different algorithms to solve a problem, we
evaluate the performance of all those algorithms. Performance evaluation aids in the selection of
the best algorithm from a set of competing algorithms for a given issue. So, we can express
algorithm performance as a practice of producing evaluation judgments regarding algorithms.

Factors Determining Algorithm’s Performance

To compare algorithms, we consider a collection of parameters or components such as the

amount of memory required by the algorithm, its execution speed, how easy it is to comprehend
Page 1
 UNIT 1

and execute, and so on. In general, an algorithm’s performance is determined by the following
factors:

 Is that algorithm giving you the perfect solution to your problem?

 Is it straightforward to comprehend?

 Is it simple to put into practice?

 How much memory (space) is needed to solve the problem?

 How long does it take to remedy the issue?

When we aim to analyze an algorithm, we just look at space and time requirements of that
algorithm and neglect anything else. On the basis of this information, an algorithm’s
performance may alternatively be described as a technique of determining the space and time
requirements of an algorithm.

The following metrics are used to evaluate the performance of an algorithm:

 The amount of space necessary to perform the algorithm’s task (Space Complexity). It
consists of both program and data space.

 The time necessary to accomplish the algorithm’s task (Time Complexity).

Space Complexity

When we create a problem-solving algorithm, it demands the use of computer memory to

complete its execution. Memory is necessary for the following purposes in any algorithm:

 To keep track of software instructions.

 To keep track of constant values.

 To keep track of variable values.

Page 2
 UNIT 1

 Additionally, for a few additional things such as function calls, jump statements, and so on.

When software is running, it often utilizes computer memory for the following reasons:

 The amount of memory needed to hold compiled versions of instructions, which is referred
to as instruction space.

 The amount of memory utilized to hold information about partially run functions at the time
of a function call, which is known as the environmental stack.

 The amount of memory needed to hold all of the variables and constants, which is referred
to as data space.

Example
We need to know how much memory is required to store distinct datatype values to compute the
space complexity (according to the compiler). Take a look at the following code:

int square(int a)

return a*a;

In the preceding piece of code, variable ‘a’ takes up 2 bytes of memory, and the return value
takes up another 2 bytes, i.e., it takes a total of 4 bytes of memory to finish its execution, and for
any input value of ‘a’, this 4 byte memory is fixed. Constant Space Complexity is the name
given to this type of space complexity.

Time Complexity

Every algorithm needs a certain amount of computer time to carry out

its instructions and complete the operation. The amount of computer time required is referred to
as time complexity. In general, an algorithm’s execution time is determined by the following:

 It doesn’t matter if it’s on a single processor or a multi-processor computer.

 It doesn’t matter if it’s a 32-bit or 64-bit computer.

Page 3
 UNIT 1

 The machine’s read and write speeds.

 The time taken by an algorithm to complete arithmetic, logical, return value, and assignment
operations, among other things.

 Data to be entered

Example
Calculating an algorithm’s Time Complexity based on the system configuration is a challenging
undertaking since the configuration varies from one system to the next. We must assume a
model machine with a certain setup to tackle this challenge. As a result, we can compute
generalized time complexity using that model machine. Take a look at the following code:

int sum(int a, int b)

return a+b;

In the following example code, calculating a+b takes 1 unit of time, and returning the value
takes 1 unit of time, i.e., it takes two units of time to perform the task, and it is unaffected by the
input values for a and b. This indicates it takes the same amount of time for all input values, i.e.,
2 units.

Notation of Performance Measurement

The Notation for Performance Measurement of an Algorithm

We must compute the complexity of an algorithm if we wish

to do an analysis on it. However, calculating an algorithm’s
complexity does not reveal the actual amount of resources
required. Rather than taking the precise quantity of resources,
we describe the complexity in a generic form (Notation),
which yields the algorithm’s essential structure. That Notation
is termed Asymptotic Notation and is a mathematical
representation of the algorithm’s complexity. The following are three asymptotic notations for
indicating time-complexity each of which is based on three separate situations, namely, the best
case, worst case, and average case:
Page 4
 UNIT 1

Big – O (Big-Oh)

The top-bound of an algorithm’s execution time is represented

by the Big-O notation. It’s the total amount of time an
algorithm takes for all input values. It indicates an algorithm’s
worst-case time complexity.

Big – Ω (Omega)

The omega notation represents the lowest bound of an

algorithm’s execution time. It specifies the shortest time an
algorithm requires for all input values. It is the best-case
scenario for the time complexity of an algorithm.

Big – Θ (Theta)

The function is enclosed in the theta notation from above and

below. It reflects the average case of an algorithm’s time
complexity and defines the upper and lower boundaries of its
execution time.

Recurrence Relation

Page 5
 UNIT 1

A recurrence is an equation or inequality that describes a function in terms of its values on smaller inputs. To
solve a Recurrence Relation means to obtain a function defined on the natural numbers that satisfy the
recurrence.

For Example, the Worst Case Running Time T(n) of the MERGE SORT Procedures is described by the
recurrence.

T (n) = θ (1) if n=1

2T + θ (n) if n>1

There are four methods for solving Recurrence:

1. Substitution Method

2. Iteration Method

3. Recursion Tree Method

4. Master Method

1. Substitution Method:
The Substitution Method Consists of two main steps:

1. Guess the Solution.

2. Use the mathematical induction to find the boundary condition and shows that the guess is correct.

For Example1 Solve the equation by Substitution Method.

T (n) = T +n

We have to show that it is asymptotically bound by O (log n).

Solution:

For T (n) = O (log n)

We have to show that for some constant c

T (n) ≤c logn.

Put this in given Recurrence Equation.

T (n) ≤c log +1

≤c log + 1 = c logn-clog2 2+1

≤c logn for c≥1

Thus T (n) =O logn. Page 6

 UNIT 1

Example2 Consider the Recurrence

T (n) = 2T + n n>1

Find an Asymptotic bound on T.

Solution:

We guess the solution is O (n (logn)).Thus for constant 'c'.

T (n) ≤c n logn

Put this in given Recurrence Equation.

Now,

T (n) ≤2c log +n

≤cnlogn-cnlog2+n

=cn logn-n (clog2-1)

≤cn logn for (c≥1)

Thus T (n) = O (n logn).

2. Iteration Methods
It means to expand the recurrence and express it as a summation of terms of n and initial condition.

Example1: Consider the Recurrence

T (n) = 1 if n=1

= 2T (n-1) if n>1

Solution:

T (n) = 2T (n-1)

= 2[2T (n-2)] = 22T (n-2)

= 4[2T (n-3)] = 23T (n-3)

= 8[2T (n-4)] = 24T (n-4) (Eq.1)

Repeat the procedure for i times

T (n) = 2i T (n-i)

Put n-i=1 or i= n-1 in (Eq.1)

T (n) = 2n-1 T (1)

Page 7
 UNIT 1

= 2n-1 .1 {T (1) =1 .....given}

= 2n-1

Example2: Consider the Recurrence

1. T (n) = T (n-1) +1 and T (1) = θ (1).

Solution:

T (n) = T (n-1) +1

= (T (n-2) +1) +1 = (T (n-3) +1) +1+1

= T (n-4) +4 = T (n-5) +1+4

= T (n-5) +5= T (n-k) + k

Where k = n-1

T (n-k) = T (1) = θ (1)

T (n) = θ (1) + (n-1) = 1+n-1=n= θ (n).

Recursion Tree Method

Recursion is a fundamental concept in computer science and mathematics that allows functions to call
themselves, enabling the solution of complex problems through iterative steps. One visual representation
commonly used to understand and analyze the execution of recursive functions is a recursion tree. In this
article, we will explore the theory behind recursion trees, their structure, and their significance in
understanding recursive algorithms.

What is a Recursion Tree?

A recursion tree is a graphical representation that illustrates the execution flow of a recursive function. It
provides a visual breakdown of recursive calls, showcasing the progression of the algorithm as it branches out
and eventually reaches a base case. The tree structure helps in analyzing the time complexity and
understanding the recursive process involved.

Tree Structure
Each node in a recursion tree represents a particular recursive call. The initial call is depicted at the top, with
subsequent calls branching out beneath it. The tree grows downward, forming a hierarchical structure. The
branching factor of each node depends on the number of recursive calls made within the function.
Additionally, the depth of the tree corresponds to the number of recursive calls before reaching the base case.

Base Case
The base case serves as the termination condition for a recursive function. It defines the point at which the
recursion stops and the function starts returning values. In a recursion tree, the nodes representing the base
case are usually depicted as leaf nodes, as they do not result in further recursive calls. Page 8
 UNIT 1

Recursive Calls
The child nodes in a recursion tree represent the recursive calls made within the function. Each child node
corresponds to a separate recursive call, resulting in the creation of new sub problems. The values or
parameters passed to these recursive calls may differ, leading to variations in the sub problems' characteristics.

Execution Flow:
Traversing a recursion tree provides insights into the execution flow of a recursive function. Starting from the
initial call at the root node, we follow the branches to reach subsequent calls until we encounter the base case.
As the base cases are reached, the recursive calls start to return, and their respective nodes in the tree are
marked with the returned values. The traversal continues until the entire tree has been traversed.

Time Complexity Analysis

Recursion trees aid in analysing the time complexity of recursive algorithms. By examining the structure of
the tree, we can determine the number of recursive calls made and the work done at each level. This analysis
helps in understanding the overall efficiency of the algorithm and identifying any potential inefficiencies or
opportunities for optimization.

Introduction
o Think of a program that determines a number's factorial. This function takes a number N as an input
and returns the factorial of N as a result. This function's pseudo-code will resemble,

// find factorial of a number

factorial(n) {
// Base case
if n is less than 2: // Factorial of 0, 1 is 1
return n

// Recursive step
return n * factorial(n-1); // Factorial of 5 => 5 * Factorial(4)...
}

/* How function calls are made,

Factorial(5) [ 120 ]
|
5 * Factorial(4) ==> 120
|
4. * Factorial(3) ==> 24
|
3 * Factorial(2) ==> 6
|
2 * Factorial(1) ==> 2
|
1
*/
Page 9
 UNIT 1

o Recursion is exemplified by the function that was previously mentioned. We are invoking a function to
determine a number's factorial. Then, given a lesser value of the same number, this function calls itself. This
continues until we reach the basic case, in which there are no more function calls.

o Recursion is a technique for handling complicated issues when the outcome is dependent on the outcomes of
smaller instances of the same issue.

o If we think about functions, a function is said to be recursive if it keeps calling itself until it reaches the base
case.

o Any recursive function has two primary components: the base case and the recursive step. We stop going to the
recursive phase once we reach the basic case. To prevent endless recursion, base cases must be properly defined
and are crucial. The definition of infinite recursion is a recursion that never reaches the base case. If a program
never reaches the base case, stack overflow will continue to occur.

Recursion Types
Generally speaking, there are two different forms of recursion:

o Linear Recursion

o Tree Recursion

o Linear Recursion

Linear Recursion
o A function that calls itself just once each time it executes is said to be linearly recursive. A nice illustration of
linear recursion is the factorial function. The name "linear recursion" refers to the fact that a linearly recursive
function takes a linear amount of time to execute.

o Take a look at the pseudo-code below:

function doSomething(n) {

// base case to stop recursion

if nis 0:

return

// here is some instructions

// recursive step

doSomething(n-1);

o If we look at the function doSomething(n), it accepts a parameter named n and does some calculations before
calling the same procedure once more but with lower values.

o When the method doSomething() is called with the argument value n, let's say that T(n) represents the total
amount of time needed to complete the computation. For this, we can also formulate a recurrence relation, T(n)
Page 10
 UNIT 1

= T(n-1) + K. K serves as a constant here. Constant K is included because it takes time for the function to
allocate or de-allocate memory to a variable or perform a mathematical operation. We use K to define the time
since it is so minute and insignificant.

o This recursive program's time complexity may be simply calculated since, in the worst scenario, the method
doSomething() is called n times. Formally speaking, the function's temporal complexity is O(N).

Tree Recursion
o When you make a recursive call in your recursive case more than once, it is referred to as tree recursion. An
effective illustration of Tree recursion is the fibonacci sequence. Tree recursive functions operate in exponential
time; they are not linear in their temporal complexity.

o Take a look at the pseudo-code below,

function doSomething(n) {

// base case to stop recursion

if n is less than 2:

return n;

// here is some instructions

// recursive step

return doSomething(n-1) + doSomething(n-2);

o The only difference between this code and the previous one is that this one makes one more call to the same
function with a lower value of n.

o Let's put T(n) = T(n-1) + T(n-2) + k as the recurrence relation for this function. K serves as a constant once
more.

o When more than one call to the same function with smaller values is performed, this sort of recursion is known
as tree recursion. The intriguing aspect is now: how time-consuming is this function?

o Take a guess based on the recursion tree below for the same function.

Page 11
 UNIT 1

o It may occur to you that it is challenging to estimate the time complexity by looking directly at a recursive
function, particularly when it is a tree recursion. Recursion Tree Method is one of several techniques for
calculating the temporal complexity of such functions. Let's examine it in further detail.

What Is Recursion Tree Method?

o Recurrence relations like T(N) = T(N/2) + N or the two we covered earlier in the kinds of recursion section are
solved using the recursion tree approach. These recurrence relations often use a divide and conquer strategy to
address problems.

o It takes time to integrate the answers to the smaller sub problems that are created when a larger problem is
broken down into smaller sub problems.

o The recurrence relation, for instance, is T(N) = 2 * T(N/2) + O(N) for the Merge sort. The time needed to
combine the answers to two sub problems with a combined size of T(N/2) is O(N), which is true at the
implementation level as well.

o For instance, since the recurrence relation for binary search is T(N) = T(N/2) + 1, we know that each iteration
of binary search results in a search space that is cut in half. Once the outcome is determined, we exit the function.
The recurrence relation has +1 added because this is a constant time operation.

o The recurrence relation T(n) = 2T(n/2) + Kn is one to consider. Kn denotes the amount of time required to
combine the answers to n/2-dimensional sub problems.

o Let's depict the recursion tree for the aforementioned recurrence relation.

We may draw a few conclusions from studying the recursion tree above, including

1. The magnitude of the problem at each level is all that matters for determining the value of a node. The issue
size is n at level 0, n/2 at level 1, n/2 at level 2, and so on.

2. In general, we define the height of the tree as equal to log (n), where n is the size of the issue, and the height
of this recursion tree is equal to the number of levels in the tree. This is true because, as we just established,
the divide-and-conquer strategy is used by recurrence relations to solve problems, and getting from issue size
n to problem size 1 simply requires taking log (n) steps.
Page 12
 UNIT 1

• Consider the value of N = 16, for instance. If we are permitted to divide N by 2 at each step,
how many steps are required to get N = 1? Considering that we are dividing by two at each
step, the correct answer is 4, which is the value of log(16) base 2.

log(16) base 2

log(2^4) base 2

4 * log(2) base 2, since log(a) base a = 1

so, 4 * log(2) base 2 = 4

3. At each level, the second term in the recurrence is regarded as the root.

Although the word "tree" appears in the name of this strategy, you don't need to be an expert on trees to
comprehend it.

How to Use a Recursion Tree to Solve Recurrence Relations?

The cost of the sub problem in the recursion tree technique is the amount of time needed to solve the sub
problem. Therefore, if you notice the phrase "cost" linked with the recursion tree, it simply refers to the amount
of time needed to solve a certain sub problem.

Let's understand all of these steps with a few examples.

Example

Consider the recurrence relation,

T(n) = 2T(n/2) + K

Solution

The given recurrence relation shows the following properties,

A problem size n is divided into two sub-problems each of size n/2. The cost of combining the solutions to
these sub-problems is K.

Each problem size of n/2 is divided into two sub-problems each of size n/4 and so on.

At the last level, the sub-problem size will be reduced to 1. In other words, we finally hit the base case.

Let's follow the steps to solve this recurrence relation,

Step 1: Draw the Recursion Tree

Page 13
 UNIT 1

Step 2: Calculate the Height of the Tree

Since we know that when we continuously divide a number by 2, there comes a time when this number is
reduced to 1. Same as with the problem size N, suppose after K divisions by 2, N becomes equal to 1, which
implies, (n / 2^k) = 1

Here n / 2^k is the problem size at the last level and it is always equal to 1.

Now we can easily calculate the value of k from the above expression by taking log() to both sides. Below is
a more clear derivation,

n = 2^k

o log(n) = log(2^k)

o log(n) = k * log(2)

o k = log(n) / log(2)

o k = log(n) base 2

So the height of the tree is log (n) base 2.

Step 3: Calculate the cost at each level

o Cost at Level-0 = K, two sub-problems are merged.

o Cost at Level-1 = K + K = 2*K, two sub-problems are merged two times.

o Cost at Level-2 = K + K + K + K = 4*K, two sub-problems are merged four times. and so on....

Step 4: Calculate the number of nodes at each level

Let's first determine the number of nodes in the last level. From the recursion tree, we can deduce this

o Level-0 have 1 (2^0) node

o Level-1 have 2 (2^1) nodes

o Level-2 have 4 (2^2) nodes Page 14

 UNIT 1

o Level-3 have 8 (2^3) nodes

So the level log(n) should have 2^(log(n)) nodes i.e. n nodes.

Step 5: Sum up the cost of all the levels

o The total cost can be written as,

o Total Cost = Cost of all levels except last level + Cost of last level

o Total Cost = Cost for level-0 + Cost for level-1 + Cost for level-2 +.... + Cost for level-log(n) + Cost
for last level

The cost of the last level is calculated separately because it is the base case and no merging is done at the last
level so, the cost to solve a single problem at this level is some constant value. Let's take it as O (1).

Let's put the values into the formulae,

o T(n) = K + 2*K + 4*K + .... + log(n)` times + `O(1) * n

o T(n) = K(1 + 2 + 4 + .... + log(n) times)` + `O(n)

o T(n) = K(2^0 + 2^1 + 2^2 + ....+ log(n) times + O(n)

If you closely take a look to the above expression, it forms a Geometric progression (a, ar, ar^2, ar^3 ......
infinite time). The sum of GP is given by S(N) = a / (r - 1). Here is the first term and r is the common ratio.

Master Method
The Master Method is used for solving the following types of recurrence

T (n) = a T + f (n) with a≥1 and b≥1 be constant & f(n) be a function and can be interpreted as

Let T (n) is defined on non-negative integers by the recurrence.

T (n) = a T + f (n)

In the function to the analysis of a recursive algorithm, the constants and function take on the following
significance:

o n is the size of the problem.

o a is the number of subproblems in the recursion.

o n/b is the size of each subproblem. (Here it is assumed that all subproblems are essentially the same size.)

o f (n) is the sum of the work done outside the recursive calls, which includes the sum of dividing the problem
and the sum of combining the solutions to the subproblems.

o It is not possible always bound the function according to the requirement, so we make three cases which will
tell us what kind of bound we can apply on the function.
Page 15
 UNIT 1

Master Theorem:
It is possible to complete an asymptotic tight bound in these three cases:

Case1: If f (n) = for some constant ε >0, then it follows that:

T (n) = Θ

Example:

T (n) = 8 T apply master theorem on it.

Solution:

Compare T (n) = 8 T with

T (n) = a T

a = 8, b=2, f (n) = 1000 n2, logba = log28 = 3

Put all the values in: f (n) =

1000 n2 = O (n3-ε )

If we choose ε=1, we get: 1000 n2 = O (n3-1) = O (n2)

Since this equation holds, the first case of the master theorem applies to the given recurrence relation, thus
resulting in the conclusion:

T (n) = Θ

Therefore: T (n) = Θ (n3)

Case 2: If it is true, for some constant k ≥ 0 that:

Page 16
 UNIT 1

F (n) = Θ then it follows that: T (n) = Θ

Example:

T (n) = 2 , solve the recurrence by using the master method.

As compare the given problem with T (n) = a T a = 2, b=2, k=0, f (n) = 10n, logba =
log22 =1

Put all the values in f (n) =Θ , we will get

10n = Θ (n1) = Θ (n) which is true.

Therefore: T (n) = Θ

= Θ (n log n)

Case 3: If it is true f(n) = Ω for some constant ε >0 and it also true that: a f for
some constant c<1 for large value of n ,then :

T (n) = Θ((f (n))

Example: Solve the recurrence relation:

T (n) = 2

Solution:

Compare the given problem with T (n) = a T

a= 2, b =2, f (n) = n2, logba = log22 =1

Put all the values in f (n) = Ω ..... (Eq. 1)

If we insert all the value in (Eq.1), we will get

n2 = Ω(n1+ε) put ε =1, then the equality will hold.

n2 = Ω(n1+1) = Ω(n2)

Now we will also check the second condition:

Page 17
 UNIT 1

If we will choose c =1/2, it is true:

∀ n ≥1

So it follows: T (n) = Θ ((f (n))

T (n) = Θ(n2)

Different types of recurrence relations and their solutions

we will see how we can solve different types of recurrence relations using different approaches.
Before understanding this article, you should have idea about recurrence relations and different
method to solve them (See : Worst, Average and Best Cases, Asymptotic Notations, Analysis of
Loops).

Type 1: Divide and conquer recurrence relations –

Following are some of the examples of recurrence relations based on divide and conquer.

T(n) = 2T(n/2) + cn

T(n) = 2T(n/2) + √n

These types of recurrence relations can be easily solved using Master Method.
For recurrence relation T(n) = 2T(n/2) + cn, the values of a = 2, b = 2 and k =1. Here logb(a) =
log2(2) = 1 = k. Therefore, the complexity will be Θ(nlog2(n)).
Similarly for recurrence relation T(n) = 2T(n/2) + √n, the values of a = 2, b = 2 and k =1/2. Here
logb(a) = log2(2) = 1 > k. Therefore, the complexity will be Θ(n).

Type 2: Linear recurrence relations –

Following are some of the examples of recurrence relations based on linear recurrence relation.

T(n) = T(n-1) + n for n>0 and T(0) = 1

These types of recurrence relations can be easily solved using substitution method.
For example,

T(n) = T(n-1) + n

= T(n-2) + (n-1) + n

= T(n-k) + (n-(k-1))….. (n-1) + n

Substituting k = n, we get
Page 18
 UNIT 1

T(n) = T(0) + 1 + 2+….. +n = n(n+1)/2 = O(n^2)

Type 3: Value substitution before solving –

Sometimes, recurrence relations can’t be directly solved using techniques like substitution,
recurrence tree or master method. Therefore, we need to convert the recurrence relation into
appropriate form before solving. For example,

T(n) = T(√n) + 1

To solve this type of recurrence, substitute n = 2^m as:

T(2^m) = T(2^m /2) + 1

Let T(2^m) = S(m),

S(m) = S(m/2) + 1

Solving by master method, we get

S(m) = Θ(logm)

As n = 2^m or m = log2(n),

T(n) = T(2^m) = S(m) = Θ(logm) = Θ(loglogn)

Let us discuss some questions based on the approaches discussed.

Que – 1. What is the time complexity of Tower of Hanoi problem?

(A) T(n) = O(sqrt(n))
(D) T(n) = O(n^2)
(C) T(n) = O(2^n)
(D) None

Solution: For Tower of Hanoi, T(n) = 2T(n-1) + c for n>1 and T(1) = 1. Solving this,

T(n) = 2T(n-1) + c

= 2(2T(n-2)+ c) + c = 2^2*T(n-2) + (c + 2c)

= 2^k*T(n-k) + (c + 2c + .. kc)

Substituting k = (n-1), we get

T(n) = 2^(n-1)*T(1) + (c + 2c + (n-1)c) = O(2^n)

Que – 2. Consider the following recurrence:

T(n) = 2 * T(ceil (sqrt(n) ) ) + 1, T(1) = 1
Which one of the following is true?
(A) T(n) = (loglogn)
(B) T(n) = (logn) Page 19
 UNIT 1

(C) T(n) = (sqrt(n))

(D) T(n) = (n)

Solution: To solve this type of recurrence, substitute n = 2^m as:

T(2^m) = 2T(2^m /2) + 1

Let T(2^m) = S(m),

S(m) = 2S(m/2) + 1

Solving by master method, we get

S(m) = Θ(m)

As n = 2^m or m = log2n,

T(n) = T(2^m) = S(m) = Θ(m) = Θ(logn)

Practice Set for Recurrence Relations

Que-1. Solve the following recurrence relation?
T(n) = 7T(n/2) + 3n^2 + 2
(a) O(n^2.8)
(b) O(n^3)
(c) θ(n^2.8)
(d) θ(n^3)

Explanation –
T(n) = 7T(n/2) + 3n^2 + 2
As one can see from the formula above:
a = 7, b = 2, and f(n) = 3n^2 + 2
So, f(n) = O(n^c), where c = 2.
It falls in master’s theorem case 1:
logb(a) = log2(7) = 2.81 > 2
It follows from the first case of the master theorem that T(n) = θ(n^2.8) and implies O(n^2.8) as
well as O(n^3).
Therefore, option (a), (b), and (c) are correct options.

Que-2. Sort the following functions in the decreasing order of their asymptotic (big-O)
complexity:
f1(n) = n^√n , f2(n) = 2^n, f3(n) = (1.000001)^n , f4(n) = n^(10)*2^(n/2)
(a) f2> f4> f1> f3
Page 20
 UNIT 1

(b) f2> f4> f3> f1

(c) f1> f2> f3> f4
(d) f2> f1> f4> f3

Explanation –
f2 > f4 because we can write f2(n) = 2^(n/2)*2^(n/2), f4(n) = n^(10)*2^(n/2) which clearly shows
that f2 > f4
f4 > f3 because we can write f4(n) = n^10.〖√2〗^n = n10.(1.414)n , which clearly shows f4> f3
f3> f1:
f1 (n) = n^√n take log both side log f1 = √n log n
f3 (n) = (1.000001)^n take log both side log f3 = n log(1.000001), we can write as log f3 = √n*√n
log(1.000001) and √n > log(1.000001).
So, correct order is f2> f4> f3> f1. Option (b) is correct.

Que-3. f(n) = 2^(2n)

Which of the following correctly represents the above function?
(a) O(2^n)
(b) Ω(2^n)
(c) Θ(2^n)
(d) None of these

Explanation – f(n) = 2^(2n) = 2^n*2^n

Option (a) says f(n)<= c*2n, which is not true. Option (c) says c1*2n <= f(n) <= c2*2n, lower
bound is satisfied but upper bound is not satisfied. Option (b) says c*2n <= f(n) this condition is
satisfied hence option (b) is correct.
Que-4. Master’s theorem can be applied on which of the following recurrence relation?
(a) T (n) = 2T (n/2) + 2^n
(b) T (n) = 2T (n/3) + sin(n)
(c) T (n) = T (n-2) + 2n^2 + 1
(d) None of these

Explanation – Master theorem can be applied to the recurrence relation of the following type
T (n) = aT(n/b) + f (n) (Dividing Function) & T(n)=aT(n-b)+f(n) (Decreasing function)
Option (a) is wrong because to apply master’s theorem, function f(n) should be polynomial.
Option (b) is wrong because in order to apply master theorem f(n) should be monotonically
increasing function.
Option (d) is not the above mentioned type, therefore correct answer is (c) because T (n) = T (n-2)
+ 2n^2 + 1 will be considered as T (n) = T (n-2) + 2n^2 that is in the form of decreasing function.
Page 21
 UNIT 1

Que-5. T(n) = 3T(n/2+ 47) + 2n^2 + 10*n – 1/2. T(n) will be

(a) O(n^2)
(b) O(n^(3/2))
(c) O(n log n)
(d) None of these

Explanation – For higher values of n, n/2 >> 47, so we can ignore 47, now T(n) will be
T(n) = 3T(n/2)+ 2*n^2 + 10*n – 1/2 = 3T(n/2)+ O(n^2)
Apply master theorem, it is case 3 of master theorem T(n) = O(n^2).
Option (a) is correct.

Page 22
 Unit 2

UNIT II
HIERARCHICAL DATA STRUCTURES
Binary Search Trees: Basics – Querying a Binary search tree – Insertion and Deletion- Red Black trees:
Properties of Red-Black Trees – Rotations – Insertion – Deletion -B-Trees: Definition of B -trees – Basic
operations on B-Trees – Deleting a key from a B-Tree- Heap – Heap Implementation – Disjoint Sets - Fibonacci
Heaps: structure – Mergeable-heap operations - Decreasing a key and deleting a node-Bounding the maximum
degree.

Binary Search tree

What is a tree?

A tree is a kind of data structure that is used to represent the data in hierarchical form.
It can be defined as a collection of objects or entities called as nodes that are linked together
to simulate a hierarchy. Tree is a non-linear data structure as the data in a tree is not stored
linearly or sequentially.

What is a Binary Search tree?

A binary search tree follows some order to arrange the elements. In a Binary search tree,
the value of left node must be smaller than the parent node, and the value of right node must
be greater than the parent node. This rule is applied recursively to the left and right subtrees of
the root.

Let's understand the concept of Binary search tree with an example.

In the above figure, we can observe that the root node is 40, and all the nodes of the left subtree
are smaller than the root node, and all the nodes of the right subtree are greater than the root
node.

Similarly, we can see the left child of root node is greater than its left child and smaller than
its right child. So, it also satisfies the property of binary search tree. Therefore, we can say that
the tree in the above image is a binary search tree.

Page 1
 Unit 2

Suppose if we change the value of node 35 to 55 in the above tree, check whether the tree will
be binary search tree or not.

In the above tree, the value of root node is 40, which is greater than its left child 30 but
smaller than right child of 30, i.e., 55. So, the above tree does not satisfy the property of Binary
search tree. Therefore, the above tree is not a binary search tree.

Advantages of Binary search tree

o Searching an element in the Binary search tree is easy as we always have a hint that
which subtree has the desired element.
o As compared to array and linked lists, insertion and deletion operations are faster in
BST.

Example of creating a binary search tree

Now, let's see the creation of binary search tree using an example.

Suppose the data elements are - 45, 15, 79, 90, 10, 55, 12, 20, 50

o First, we have to insert 45 into the tree as the root of the tree.
o Then, read the next element; if it is smaller than the root node, insert it as the root of the
left subtree, and move to the next element.
o Otherwise, if the element is larger than the root node, then insert it as the root of the
right subtree.

Now, let's see the process of creating the Binary search tree using the given data element. The
process of creating the BST is shown below –

Step 1 - Insert 45.

Page 2
 Unit 2

Step 2 - Insert 15.

As 15 is smaller than 45, so insert it as the root node of the left subtree.

Step 3 - Insert 79.

As 79 is greater than 45, so insert it as the root node of the right subtree.

Step 4 - Insert 90.

90 is greater than 45 and 79, so it will be inserted as the right subtree of 79.

Page 3
 Unit 2

Step 5 - Insert 10.

10 is smaller than 45 and 15, so it will be inserted as a left subtree of 15.

Step 6 - Insert 55.

55 is larger than 45 and smaller than 79, so it will be inserted as the left subtree of 79.

Step 7 - Insert 12.

12 is smaller than 45 and 15 but greater than 10, so it will be inserted as the right subtree of
10.

Page 4
 Unit 2

Step 8 - Insert 20.

20 is smaller than 45 but greater than 15, so it will be inserted as the right subtree of 15.

Step 9 - Insert 50.

50 is greater than 45 but smaller than 79 and 55. So, it will be inserted as a left subtree of 55.

Now, the creation of binary search tree is completed. After that, let's move towards the
operations that can be performed on Binary search tree.

We can perform insert, delete and search operations on the binary search tree.

Let's understand how a search is performed on a binary search tree.

Searching in Binary search tree

Searching means to find or locate a specific element or node in a data structure. In Binary
search tree, searching a node is easy because elements in BST are stored in a specific order.
The steps of searching a node in Binary Search tree are listed as follows -

1. First, compare the element to be searched with the root element of the tree.
Page 5
 Unit 2

2. If root is matched with the target element, then return the node's location.
3. If it is not matched, then check whether the item is less than the root element, if it is
smaller than the root element, then move to the left subtree.
4. If it is larger than the root element, then move to the right subtree.
5. Repeat the above procedure recursively until the match is found.
6. If the element is not found or not present in the tree, then return NULL.

Now, let's understand the searching in binary tree using an example. We are taking the binary
search tree formed above. Suppose we have to find node 20 from the below tree.

Step1:

Step2:

Page 6
 Unit 2

Step3:

Now, let's see the algorithm to search an element in the Binary search tree.

Algorithm to search an element in Binary search tree

Search (root, item)
Step 1 - if (item = root → data) or (root = NULL)
return root
else if (item < root → data)
return Search (root → left, item)
else
return Search (root → right, item)
END if
Step 2 - END

Now let's understand how the deletion is performed on a binary search tree. We will also see
an example to delete an element from the given tree.

Deletion in Binary Search tree

In a binary search tree, we must delete a node from the tree by keeping in mind that the property
of BST is not violated. To delete a node from BST, there are three possible situations occur -

o The node to be deleted is the leaf node, or,

o The node to be deleted has only one child, and,
o The node to be deleted has two children

When the node to be deleted is the leaf node

Page 7
 Unit 2

It is the simplest case to delete a node in BST. Here, we have to replace the leaf node
with NULL and simply free the allocated space.

We can see the process to delete a leaf node from BST in the below image. In below image,
suppose we have to delete node 90, as the node to be deleted is a leaf node, so it will be replaced
with NULL, and the allocated space will free.

When the node to be deleted has only one child

In this case, we have to replace the target node with its child, and then delete the child
node. It means that after replacing the target node with its child node, the child node will now
contain the value to be deleted. So, we simply have to replace the child node with NULL and
free up the allocated space.

We can see the process of deleting a node with one child from BST in the below image. In the
below image, suppose we have to delete the node 79, as the node to be deleted has only one
child, so it will be replaced with its child 55.

So, the replaced node 79 will now be a leaf node that can be easily deleted.

When the node to be deleted has two children

Page 8
 Unit 2

This case of deleting a node in BST is a bit complex among other two cases. In such a case,
the steps to be followed are listed as follows -

o First, find the in-order successor of the node to be deleted.

o After that, replace that node with the in-order successor until the target node is placed
at the leaf of tree.
o And at last, replace the node with NULL and free up the allocated space.

The in-order successor is required when the right child of the node is not empty. We can obtain
the in-order successor by finding the minimum element in the right child of the node.

We can see the process of deleting a node with two children from BST in the below image. In
the below image, suppose we have to delete node 45 that is the root node, as the node to be
deleted has two children, so it will be replaced with its in order successor. Now, node 45 will
be at the leaf of the tree so that it can be deleted easily.

Now let's understand how insertion is performed on a binary search tree.

Insertion in Binary Search tree

A new key in BST is always inserted at the leaf. To insert an element in BST, we have
to start searching from the root node; if the node to be inserted is less than the root node, then
search for an empty location in the left subtree. Else, search for the empty location in the right
subtree and insert the data. Insert in BST is similar to searching, as we always have to maintain
the rule that the left subtree is smaller than the root, and right subtree is larger than the root.

Now, let's see the process of inserting a node into BST using an example.

Page 9
 Unit 2

The complexity of the Binary Search tree

Let's see the time and space complexity of the Binary search tree. We will see the time
complexity for insertion, deletion, and searching operations in best case, average case, and
worst case.

Page 10
 Unit 2

1. Time Complexity
Operations Best case time Average case Worst case time
complexity time complexity complexity

Insertion O(log n) O(log n) O(n)

Deletion O(log n) O(log n) O(n)

Search O(log n) O(log n) O(n)

Where 'n' is the number of nodes in the given tree.

2. Space Complexity
Operations Space complexity

Insertion O(n)

Deletion O(n)

Search O(n)

o The space complexity of all operations of Binary search tree is O(n).

Implementation of Binary search tree

Now, let's see the program to implement the operations of Binary Search tree.

Program: Write a program to perform operations of Binary Search tree in C++.

In this program, we will see the implementation of the operations of binary search tree.
Here, we will see the creation, in-order traversal, insertion, and deletion operations of tree.

Here, we will see the in-order traversal of the tree to check whether the nodes of the tree
are in their proper location or not. We know that the in-order traversal always gives us the data
in ascending order. So, after performing the insertion and deletion operations, we perform the
in-order traversal, and after traversing, if we get data in ascending order, then it is clear that
the nodes are in their proper location.

Page 11
 Unit 2

#include <iostream>
using namespace std;
struct Node {
int data;
Node *left;
Node *right;
};
Node* create(int item)
{
Node* node = new Node;
node->data = item;
node->left = node->right = NULL;
return node;
}
/*Inorder traversal of the tree formed*/
void inorder(Node *root)
{
if (root == NULL)
return;
inorder(root->left); //traverse left subtree
cout<< root->data << " "; //traverse root node
inorder(root->right); //traverse right subtree
}
Node* findMinimum(Node* cur) /*To find the inorder successor*/
{
while(cur->left != NULL) {
cur = cur->left;
}
return cur;
}
Node* insertion(Node* root, int item) /*Insert a node*/
{
if (root == NULL)
return create(item); /*return new node if tree is empty*/
if (item < root->data)
root->left = insertion(root->left, item);
else
root->right = insertion(root->right, item);
Page 12
 Unit 2

return root;
}
void search(Node* &cur, int item, Node* &parent)
{
while (cur != NULL && cur->data != item)
{
parent = cur;
if (item < cur->data)
cur = cur->left;
else
cur = cur->right;
}
}
void deletion(Node*& root, int item) /*function to delete a node*/
{
Node* parent = NULL;
Node* cur = root;
search(cur, item, parent); /*find the node to be deleted*/
if (cur == NULL)
return;
if (cur->left == NULL && cur->right == NULL) /*When node has no children*/
{
if (cur != root)
{
if (parent->left == cur)
parent->left = NULL;
else
parent->right = NULL;
}
else
root = NULL;
free(cur);
}
else if (cur->left && cur->right)
{
Node* succ = findMinimum(cur->right);
int val = succ->data;
deletion(root, succ->data);
Page 13
 Unit 2

cur->data = val;
}
else
{
Node* child = (cur->left)? cur->left: cur->right;
if (cur != root)
{
if (cur == parent->left)
parent->left = child;
else
parent->right = child;
}
else
root = child;
free(cur);
}
}
int main()
{
Node* root = NULL;
root = insertion(root, 45);
root = insertion(root, 30);
root = insertion(root, 50);
root = insertion(root, 25);
root = insertion(root, 35);
root = insertion(root, 45);
root = insertion(root, 60);
root = insertion(root, 4);
printf("The inorder traversal of the given binary tree is - \n");
inorder(root);
deletion(root, 25);
printf("\nAfter deleting node 25, the inorder traversal of the given binary tree is - \n")
;
inorder(root);
insertion(root, 2);
printf("\nAfter inserting node 2, the inorder traversal of the given binary tree is - \n");

inorder(root);
Page 14
 Unit 2

return 0;
}

Output

After the execution of the above code, the output will be -

Heap Data Structure

What is Heap?

A Heap is a special Tree-based data structure in which the tree is a complete binary tree.

A heap is a complete binary tree, and the binary tree is a tree in which the node can have utmost
two children. Before knowing more about the heap data structure, we should know about the
complete binary tree.

Operations of Heap Data Structure:

• Heapify: a process of creating a heap from an array.

• Insertion: process to insert an element in existing heap time complexity O(log N).
• Deletion: deleting the top element of the heap or the highest priority element, and then organizing the heap and
returning the element with time complexity O(log N).
• Peek: to check or find the first (or can say the top) element of the heap.

Types of Heap Data Structure

Generally, Heaps can be of two types:

1. Max-Heap: In a Max-Heap the key present at the root node must be greatest among the keys present at all of
it’s children. The same property must be recursively true for all sub-trees in that Binary Tree.
2. Min-Heap: In a Min-Heap the key present at the root node must be minimum among the keys present at all of
it’s children. The same property must be recursively true for all sub-trees in that Binary Tree.

What is a complete binary tree?

A complete binary tree is a binary tree in which all the levels except the last level, i.e., leaf node
should be completely filled, and all the nodes should be left-justified.

Page 15
 Unit 2

Let's understand through an example.

In the above figure, we can observe that all the internal nodes are completely filled except the leaf
node; therefore, we can say that the above tree is a complete binary tree.

The above figure shows that all the internal nodes are completely filled except the leaf node, but the
leaf nodes are added at the right part; therefore, the above tree is not a complete binary tree.

Note: The heap tree is a special balanced binary tree data structure where the root node is compared
with its children and arrange accordingly.

How can we arrange the nodes in the Tree?

There are two types of the heap:

o Min Heap
o Max heap

Min Heap: The value of the parent node should be less than or equal to either of its children.

Or

In other words, the min-heap can be defined as, for every node i, the value of node i is greater than or equal
to its parent value except the root node. Mathematically, it can be defined as:

Page 16
 Unit 2

A[Parent(i)] <= A[i]

Let's understand the min-heap through an example.

In the above figure, 11 is the root node, and the value of the root node is less than the value of all the other
nodes (left child or a right child).

Max Heap: The value of the parent node is greater than or equal to its children.

Or

In other words, the max heap can be defined as for every node i; the value of node i is less than or equal to
its parent value except the root node. Mathematically, it can be defined as:

A[Parent(i)] >= A[i]

The above tree is a max heap tree as it satisfies the property of the max heap. Now, let's see the
array representation of the max heap.

Time complexity in Max Heap

The total number of comparisons required in the max heap is according to the height of the tree.
The height of the complete binary tree is always logn; therefore, the time complexity would also be
O(logn).

Algorithm of insert operation in the max heap.

Page 17
 Unit 2

// algorithm to insert an element in the max heap.

insertHeap(A, n, value)
{
n=n+1; // n is incremented to insert the new element
A[n]=value; // assign new value at the nth position
i = n; // assign the value of n to i
// loop will be executed until i becomes 1.
while(i>1)
{
parent= floor value of i/2; // Calculating the floor value of i/2
// Condition to check whether the value of parent is less than the given node or not
if(A[parent]<A[i])
{
swap(A[parent], A[i]);
i = parent;
}
else
{
return;
}
}
}

Let's understand the max heap through an example.

In the above figure, 55 is the parent node and it is greater than both of its child, and 11 is the parent
of 9 and 8, so 11 is also greater than from both of its child. Therefore, we can say that the above tree
is a max heap tree.

Insertion in the Heap tree

44, 33, 77, 11, 55, 88, 66

Suppose we want to create the max heap tree. To create the max heap tree, we need to consider the
following two cases:

o First, we have to insert the element in such a way that the property of the complete binary
tree must be maintained.
o Secondly, the value of the parent node should be greater than the either of its child.

Step 1: First we add the 44 element in the tree as shown below:

Page 18
 Unit 2

Step 2: The next element is 33. As we know that insertion in the binary tree always starts from the
left side so 44 will be added at the left of 33 as shown below:

Step 3: The next element is 77 and it will be added to the right of the 44 as shown below:

As we can observe in the above tree that it does not satisfy the max heap property, i.e., parent node
44 is less than the child 77. So, we will swap these two values as shown below:

Step 4: The next element is 11. The node 11 is added to the left of 33 as shown below:

Page 19
 Unit 2

Step 5: The next element is 55. To make it a complete binary tree, we will add the node 55 to the
right of 33 as shown below:

As we can observe in the above figure that it does not satisfy the property of the max heap because
33<55, so we will swap these two values as shown below:

Step 6: The next element is 88. The left subtree is completed so we will add 88 to the left of 44 as
shown below:

As we can observe in the above figure that it does not satisfy the property of the max heap because
44<88, so we will swap these two values as shown below:

Page 20
 Unit 2

Again, it is violating the max heap property because 88>77 so we will swap these two values as
shown below:

Step 7: The next element is 66. To make a complete binary tree, we will add the 66 element to the
right side of 77 as shown below:

In the above figure, we can observe that the tree satisfies the property of max heap; therefore, it is
a heap tree.

Deletion in Heap Tree

In Deletion in the heap tree, the root node is always deleted and it is replaced with the last element.

Let's understand the deletion through an example.

Step 1: In the above tree, the first 30 node is deleted from the tree and it is replaced with the 15
element as shown below:

Now we will heapify the tree. We will check whether the 15 is greater than either of its child or not.
15 is less than 20 so we will swap these two values as shown below:

Again, we will compare 15 with its child. Since 15 is greater than 10 so no swapping will occur.

Algorithm to heapify the tree

MaxHeapify(A, n, i)
{
int largest =i;
int l= 2i;
int r= 2i+1;
while(l<=n && A[l]>A[largest])
{
largest=l;
}
while(r<=n && A[r]>A[largest])
{
largest=r;
}
if(largest!=i)
{
swap(A[largest], A[i]);
heapify(A, n, largest); }}

Page 21
 UNIT 3

UNIT III GRAPHS

UNIT III GRAPHS : Elementary Graph Algorithms: Representations of Graphs – Breadth-
First Search – Depth-First Search – Topological Sort – Strongly Connected Components-
Minimum Spanning Trees: Growing a Minimum Spanning Tree – Kruskal and Prim- Single-
Source Shortest Paths: The Bellman-Ford algorithm – Single-Source Shortest paths in
Directed Acyclic Graphs – Dijkstra‘s Algorithm; Dynamic Programming - All-Pairs Shortest
Paths: Shortest Paths and Matrix Multiplication – The Floyd-Warshall Algorithm

Elementary Graph Algorithms

Introduction to Graphs
Graph is a non-linear data structure. It contains a set of points known as nodes (or vertices) and a set of links known as
edges (or Arcs). Here edges are used to connect the vertices. A graph is defined as follows...
Graph is a collection of vertices and arcs in which vertices are connected with arcs

Graph is a collection of nodes and edges in which nodes are connected with edges
Generally, a graph G is represented as G = ( V , E ), where V is set of vertices and E is set of edges.
Example
The following is a graph with 5 vertices and 6 edges.
This graph G can be defined as G = ( V , E )
Where V = {A,B,C,D,E} and E = {(A,B),(A,C)(A,D),(B,D),(C,D),(B,E),(E,D)}.

Graph Terminology
We use the following terms in graph data structure...

Vertex
Individual data element of a graph is called as Vertex. Vertex is also known as node. In above example graph, A, B, C,
D & E are known as vertices.

Edge
An edge is a connecting link between two vertices. Edge is also known as Arc. An edge is represented as
(startingVertex, endingVertex). For example, in above graph the link between vertices A and B is represented as (A,B).
In above example graph, there are 7 edges (i.e., (A,B), (A,C), (A,D), (B,D), (B,E), (C,D), (D,E)).

Edges are three types.

Undirected Edge - An undirected egde is a bidirectional edge. If there is undirected edge between vertices A and
B then edge (A , B) is equal to edge (B , A).

Page 1
 UNIT 3

Directed Edge - A directed egde is a unidirectional edge. If there is directed edge between vertices A and B then
edge (A , B) is not equal to edge (B , A).

Weighted Edge - A weighted egde is a edge with value (cost) on it.

Undirected Graph
A graph with only undirected edges is said to be undirected graph.

Directed Graph
A graph with only directed edges is said to be directed graph.

Mixed Graph
A graph with both undirected and directed edges is said to be mixed graph.

End vertices or Endpoints

The two vertices joined by edge are called end vertices (or endpoints) of that edge.

Origin
If a edge is directed, its first endpoint is said to be the origin of it.

Destination
If a edge is directed, its first endpoint is said to be the origin of it and the other endpoint is said to be the destination of
that edge.

Adjacent
If there is an edge between vertices A and B then both A and B are said to be adjacent. In other words, vertices A and
B are said to be adjacent if there is an edge between them.

Incident
Edge is said to be incident on a vertex if the vertex is one of the endpoints of that edge.

Outgoing Edge
A directed edge is said to be outgoing edge on its origin vertex.

Incoming Edge
A directed edge is said to be incoming edge on its destination vertex.

Degree
Total number of edges connected to a vertex is said to be degree of that vertex.

Indegree
Total number of incoming edges connected to a vertex is said to be indegree of that vertex.

Outdegree
Total number of outgoing edges connected to a vertex is said to be outdegree of that vertex.

Parallel edges or Multiple edges

If there are two undirected edges with same end vertices and two directed edges with same origin and destination, such
edges are called parallel edges or multiple edges.

Page 2
 UNIT 3

Self-loop
Edge (undirected or directed) is a self-loop if its two endpoints coincide with each other.

Simple Graph
A graph is said to be simple if there are no parallel and self-loop edges.

Path
A path is a sequence of alternate vertices and edges that starts at a vertex and ends at other vertex such that each edge is
incident to its predecessor and successor vertex.
Elementary Graph Algorithms

Graph Representations
Graph data structure is represented using following representations...

1. Adjacency Matrix
2. Incidence Matrix
3. Adjacency List

1. Adjacency Matrix
In this representation, the graph is represented using a matrix of size total number of vertices by a total number of
vertices. That means a graph with 4 vertices is represented using a matrix of size 4X4. In this matrix, both rows and
columns represent vertices. This matrix is filled with either 1 or 0. Here, 1 represents that there is an edge from row
vertex to column vertex and 0 represents that there is no edge from row vertex to column vertex.

For example, consider the following undirected graph representation...

Directed graph
representation...

Page 3
 UNIT 3

2. Incidence Matrix
In this representation, the graph is represented using a matrix of size total number of vertices by a total number of edges.
That means graph with 4 vertices and 6 edges is represented using a matrix of size 4X6. In this matrix, rows represent
vertices and columns represents edges. This matrix is filled with 0 or 1 or -1. Here, 0 represents that the row edge is not
connected to column vertex, 1 represents that the row edge is connected as the outgoing edge to column vertex and -1
represents that the row edge is connected as the incoming edge to column vertex.

For example, consider the following directed graph representation..

3. Adjacency List
In this representation, every vertex of a graph contains list of its adjacent vertices.

For example, consider the following directed graph representation implemented using linked list...

This representation can also be implemented using an array as follows..

Graph Traversal
Graph traversal is a technique used for a searching vertex in a graph. The graph traversal is also used to decide the order
of vertices is visited in the search process. A graph traversal finds the edges to be used in the search process without
creating loops. That means using graph traversal we visit all the vertices of the graph without getting into looping path.

There are two graph traversal techniques and they are as follows...

1. DFS (Depth First Search)

2. BFS (Breadth First Search)

DFS (Depth First Search)

Page 4
 UNIT 3

DFS traversal of a graph produces a spanning tree as final result. Spanning Tree is a graph without loops. We
use Stack data structure with maximum size of total number of vertices in the graph to implement DFS traversal.

We use the following steps to implement DFS traversal...

Page 5
UNIT 3

Page 6
UNIT 3

Page 7
 UNIT 3

• Step 1 - Define a Stack of size total number of vertices in the graph

• Step 2 - Select any vertex as starting point for traversal. Visit that vertex and push it on to the Stack.
• Step 3 - Visit any one of the non-visited adjacent vertices of a vertex which is at the top of stack and push it
on to the stack.
• Step 4 - Repeat step 3 until there is no new vertex to be visited from the vertex which is at the top of the stack.
• Step 5 - When there is no new vertex to visit then use back tracking and pop one vertex from the stack.
• Step 6 - Repeat steps 3, 4 and 5 until stack becomes Empty.
• Step 7 - When stack becomes Empty, then produce final spanning tree by removing unused edges from the
graph

Back tracking is coming back to the vertex from which we reached the current vertex.

1. BFS (Breadth First Search)

BFS traversal of a graph produces a spanning tree as final result. Spanning Tree is a graph without loops. We
use Queue data structure with maximum size of total number of vertices in the graph to implement BFS traversal.

We use the following steps to implement BFS traversal...

• Step 1 - Define a Queue of size total number of vertices in the graph.

• Step 2 - Select any vertex as starting point for traversal. Visit that vertex and insert it into the Queue.
• Step 3 - Visit all the non-visited adjacent vertices of the vertex which is at front of the Queue and insert them
into the Queue.
• Step 4 - When there is no new vertex to be visited from the vertex which is at front of the Queue then delete
that vertex.
• Step 5 - Repeat steps 3 and 4 until queue becomes empty.
• Step 6 - When queue becomes empty, then produce final spanning tree by removing unused edges from the
graph

Page 8
 UNIT 3

Topological Sort

Topological Sort is a linear ordering of the vertices in such a way that

if there is an edge in the DAG going from vertex ‘u’ to vertex ‘v’,
then ‘u’ comes before ‘v’ in the ordering.

It is important to note that-

• Topological Sorting is possible if and only if the graph is a Directed Acyclic Graph.
• There may exist multiple different topological orderings for a given directed acyclic graph.

Page 9
 UNIT 3

1. Topological Sort Example

Consider the following directed acyclic graph-

For this graph, following 4 different topological orderings are possible-

• 123456
• 123465
• 132456
• 132465
Applications of Topological Sort-
Few important applications of topological sort are-
• Scheduling jobs from the given dependencies among jobs
• Instruction Scheduling
• Determining the order of compilation tasks to perform in makefiles
• Data Serialization
• PRACTICE PROBLEMS BASED ON TOPOLOGICAL SORT-
Problem-01:

Find the number of different topological orderings possible for the given graph-

1. Solution-

Page 10
 UNIT 3

The topological orderings of the above graph are found in the following steps-
Step-01: Write in-degree of each vertex-

Step-02:
• Vertex-A has the least in-degree.
• So, remove vertex-A and its associated edges.
• Now, update the in-degree of other vertices.

Step-03:
• Vertex-B has the least in-degree.
• So, remove vertex-B and its associated edges.
• Now, update the in-degree of other vertices.

Page 11
 UNIT 3

Step-04:
There are two vertices with the least in-degree. So, following 2 cases are possible-
In case-01,
• Remove vertex-C and its associated edges.
• Then, update the in-degree of other vertices.
In case-02,
• Remove vertex-D and its associated edges.
• Then, update the in-degree of other vertices.

Step-05:
Now, the above two cases are continued separately in the similar manner.
In case-01,
• Remove vertex-D since it has the least in-degree.
• Then, remove the remaining vertex-E.
In case-02,
• Remove vertex-C since it has the least in-degree.
• Then, remove the remaining vertex-E.

Conclusion
For the given graph, following 2 different topological orderings are possible-
• ABCDE
• ABDCE

Strongly Connected Components

Page 12
 UNIT 3

A strongly connected component is the portion of a directed graph in which there is a path from each vertex to another
vertex. It is applicable only on a directed graph.
For example:

Let us take the graph below.

The strongly connected components of the above graph are:

Strongly connected components

You can observe that in the first strongly connected component, every vertex can reach the other vertex through the
directed path.

These components can be found using Kosaraju's Algorithm.

Kosaraju's Algorithm

Kosaraju's Algorithm is based on the depth-first search algorithm implemented twice.

Three steps are involved.

Perform a depth first search on the whole graph.

Let us start from vertex-0, visit all of its child vertices, and mark the visited vertices as done. If a vertex leads to an
already visited vertex, then push this vertex to the stack.

Page 13
 UNIT 3

For example: Starting from vertex-0, go to vertex-1, vertex-2, and then to vertex-3. Vertex-3 leads to already visited
vertex-0, so push the source vertex (ie. vertex-3) into the stack.

DFS on the graph

Go to the previous vertex (vertex-2) and visit its child vertices i.e. vertex-4, vertex-5, vertex-6 and vertex-7
sequentially. Since there is nowhere to go from vertex-7, push it into the stack

DFS on the graph

Go to the previous vertex (vertex-6) and visit its child vertices. But, all of its child vertices are visited, so push it into
the stack

Page 14
 UNIT 3

. Stacking

Similarly, a final stack is created.

Final Stack
Reverse the original graph

Perform depth-first search on the reversed graph.

Start from the top vertex of the stack. Traverse through all of its child vertices. Once the already visited vertex is
reached, one strongly connected component is formed.

For example: Pop vertex-0 from the stack. Starting from vertex-0, traverse through its child vertices (vertex-0, vertex-
1, vertex-2, vertex-3 in sequence) and mark them as visited. The child of vertex-3 is already visited, so these visited
vertices form one strongly connected component

Start from the top and traverse through all the vertices
Page 15
 UNIT 3

Go to the stack and pop the top vertex if already visited. Otherwise, choose the top vertex from the stack and traverse
through its child vertices as presented above.

Minimum Spanning Tree

Before knowing about the minimum spanning tree, we should know about the spanning tree.

To understand the concept of spanning tree, consider the below graph:

Page 16
 UNIT 3

The above graph can be represented as G(V, E), where 'V' is the number of vertices, and 'E' is the number of edges. The
spanning tree of the above graph would be represented as G`(V`, E`). In this case, V` = V means that the number of
vertices in the spanning tree would be the same as the number of vertices in the graph, but the number of edges would
be different. The number of edges in the spanning tree is the subset of the number of edges in the original graph.
Therefore, the number of edges can be written as:

E` € E

It can also be written as:

E` = |V| - 1

Two conditions exist in the spanning tree, which is as follows:

o The number of vertices in the spanning tree would be the same as the number of vertices in the original graph.
V` = V
o The number of edges in the spanning tree would be equal to the number of edges minus 1.
E` = |V| - 1
o The spanning tree should not contain any cycle.
o The spanning tree should not be disconnected.
o A graph can have more than one spanning tree.

Consider the below graph:

The above graph contains 5 vertices. As we know, the vertices in the spanning tree would be the same as the graph;
therefore, V` is equal 5. The number of edges in the spanning tree would be equal to (5 - 1), i.e., 4. The following are
the possible spanning trees:

Page 17
 UNIT 3

What is a minimum spanning tree?

The minimum spanning tree is a spanning tree whose sum of the edges is minimum. Consider the below graph that
contains the edge weight:

The following are the spanning trees that we can make from the above graph.

o The first spanning tree is a tree in which we have removed the edge between the vertices 1 and 5 shown as
below:
The sum of the edges of the above tree is (1 + 4 + 5 + 2): 12
o The second spanning tree is a tree in which we have removed the edge between the vertices 1 and 2 shown as
below:
The sum of the edges of the above tree is (3 + 2 + 5 + 4) : 14
o The third spanning tree is a tree in which we have removed the edge between the vertices 2 and 3 shown as
below:
The sum of the edges of the above tree is (1 + 3 + 2 + 5) : 11
o The fourth spanning tree is a tree in which we have removed the edge between the vertices 3 and 4 shown as
below:
The sum of the edges of the above tree is (1 + 3 + 2 + 4) : 10. The edge cost 10 is minimum so it is a minimum
spanning tree.

General properties of minimum spanning tree:

Page 18
 UNIT 3

o If we remove any edge from the spanning tree, then it becomes disconnected. Therefore, we cannot remove any
edge from the spanning tree.
o If we add an edge to the spanning tree then it creates a loop. Therefore, we cannot add any edge to the spanning
tree.
o In a graph, each edge has a distinct weight, then there exists only a single and unique minimum spanning tree.
If the edge weight is not distinct, then there can be more than one minimum spanning tree.
o A complete undirected graph can have an nn-2 number of spanning trees.
o Every connected and undirected graph contains atleast one spanning tree.
o The disconnected graph does not have any spanning tree.
o In a complete graph, we can remove maximum (e-n+1) edges to construct a spanning tree.

Let's understand the last property through an example.

Consider the complete graph which is given below:

The number of spanning trees that can be made from the above complete graph equals to nn-2 = 44-2 = 16.

Therefore, 16 spanning trees can be created from the above graph.

The maximum number of edges that can be removed to construct a spanning tree equals to e-n+1 = 6 - 4 + 1 = 3.

Application of Minimum Spanning Tree

1. Consider n stations are to be linked using a communication network & laying of communication links between
any two stations involves a cost.
The ideal solution would be to extract a subgraph termed as minimum cost spanning tree.
2. Suppose you want to construct highways or railroads spanning several cities then we can use the concept of
minimum spanning trees.
3. Designing Local Area Networks.
4. Laying pipelines connecting offshore drilling sites, refineries and consumer markets.
5. Suppose you want to apply a set of houses with
o Electric Power
o Water
o Telephone lines
o Sewage lines
Page 19
 UNIT 3

To reduce cost, you can connect houses with minimum cost spanning trees.

Prim’s and Kruskal’s Algorithms-

We have discussed-
• Prim’s and Kruskal’s Algorithm are the famous greedy algorithms.
• They are used for finding the Minimum Spanning Tree (MST) of a given graph.
• To apply these algorithms, the given graph must be weighted, connected and undirected.
Some important concepts based on them are-
Concept-01:
If all the edge weights are distinct, then both the algorithms are guaranteed to find the same MST.
Example-

Consider the following example-

Here, both the algorithms on the above given graph produces the same MST as shown.
Concept-02:
• If all the edge weights are not distinct, then both the algorithms may not always produce the same MST.
• However, cost of both the MSTs would always be same in both the cases.
Example-
Consider the following example-

Page 20
 UNIT 3

Here, both the algorithms on the above given graph produces different MSTs as shown but the cost is same in both the
cases.
Concept-03:
Kruskal’s Algorithm is preferred when-
• The graph is sparse.
• There are less number of edges in the graph like E = O(V)
• The edges are already sorted or can be sorted in linear time.

Prim’s Algorithm is preferred when-

• The graph is dense.
• There are large number of edges in the graph like E = O(V2).
Concept-04:
Difference between Prim’s Algorithm and Kruskal’s Algorithm-

Prim’s Algorithm Kruskal’s Algorithm

The tree that we are making or growing The tree that we are making or growing
always remains connected. usually remains disconnected.

Kruskal’s Algorithm grows a solution

Prim’s Algorithm grows a solution from
from the cheapest edge by adding the
a random vertex by adding the next
next cheapest edge to the existing tree /
cheapest vertex to the existing tree.
forest.

Prim’s Algorithm is faster for dense Kruskal’s Algorithm is faster for sparse
graphs. graphs.

Prim’s Algorithm-

• Prim’s Algorithm is a famous greedy algorithm.

• It is used for finding the Minimum Spanning Tree (MST) of a given graph.
• To apply Prim’s algorithm, the given graph must be weighted, connected and undirected.
Prim’s Algorithm Implementation-
The implementation of Prim’s Algorithm is explained in the following steps-
Step-01:
• Randomly choose any vertex.
• The vertex connecting to the edge having least weight is usually selected.
Step-02
• Find all the edges that connect the tree to new vertices.
• Find the least weight edge among those edges and include it in the existing tree.
• If including that edge creates a cycle, then reject that edge and look for the next least weight edge.
Step-03:
• Keep repeating step-02 until all the vertices are included and Minimum Spanning Tree (MST) is obtained.
Page 21
 UNIT 3

Prim’s Algorithm Time Complexity-

Worst case time complexity of Prim’s Algorithm is-
• O(ElogV) using binary heap
• O(E + VlogV) using Fibonacci heap
Time Complexity Analysis
• If adjacency list is used to represent the graph, then using breadth first search, all the vertices can be traversed in
O(V + E) time.
• We traverse all the vertices of graph using breadth first search and use a min heap for storing the vertices not yet
included in the MST.
• To get the minimum weight edge, we use min heap as a priority queue.
• Min heap operations like extracting minimum element and decreasing key value takes O(logV) time.

So, overall time complexity

= O(E + V) x O(logV)
= O((E + V)logV)
= O(ElogV)

This time complexity can be improved and reduced to O(E + VlogV) using Fibonacci heap.

PRACTICE PROBLEMS BASED ON PRIM’S ALGORITHM-

Problem-01:
Construct the minimum spanning tree (MST) for the given graph using Prim’s Algorithm-

Solution-

The above discussed steps are followed to find the minimum cost spanning tree using Prim’s Algorithm-

Page 22
 UNIT 3

Since all the vertices have been included in the MST, so we stop.
Now, Cost of Minimum Spanning Tree
= Sum of all edge weights
= 10 + 25 + 22 + 12 + 16 + 14
= 99 units

Kruskal’s Algorithm-
• Kruskal’s Algorithm is a famous greedy algorithm.
• It is used for finding the Minimum Spanning Tree (MST) of a given graph.
• To apply Kruskal’s algorithm, the given graph must be weighted, connected and undirected.
Kruskal’s Algorithm Implementation-
The implementation of Kruskal’s Algorithm is explained in the following steps-
Step-01:
• Sort all the edges from low weight to high weight.
Step-02:
• Take the edge with the lowest weight and use it to connect the vertices of graph.
• If adding an edge creates a cycle, then reject that edge and go for the next least weight edge.
Step-03:
• Keep adding edges until all the vertices are connected and a Minimum Spanning Tree (MST) is obtained.

Thumb Rule to Remember

The above steps may be reduced to the following thumb rule-
• Simply draw all the vertices on the paper.
• Connect these vertices using edges with minimum weights such that no cycle
gets formed.

Kruskal’s Algorithm Time Complexity-

Page 23
 UNIT 3

Worst case time complexity of Kruskal’s Algorithm

= O(ElogV) or O(ElogE)

Analysis-
• The edges are maintained as min heap.
• The next edge can be obtained in O(logE) time if graph has E edges.
• Reconstruction of heap takes O(E) time.
• So, Kruskal’s Algorithm takes O(ElogE) time.
• The value of E can be at most O(V2).
• So, O(logV) and O(logE) are same.

Special Case-
• If the edges are already sorted, then there is no need to construct min heap.
• So, deletion from min heap time is saved.
• In this case, time complexity of Kruskal’s Algorithm = O(E + V)
PRACTICE PROBLEMS BASED ON KRUSKAL’S ALGORITHM-
Problem-01:
Construct the minimum spanning tree (MST) for the given graph using Kruskal’s Algorithm-

Solution-
To construct MST using Kruskal’s Algorithm,
• Simply draw all the vertices on the paper.
• Connect these vertices using edges with minimum weights such that no cycle gets formed.

(i) (ii)

Page 24
 UNIT 3

(iii) (iv)

(v) (vi)

(vii)
Since all the vertices have been connected / included in the MST, so we stop.
Weight of the MST
= Sum of all edge weights
= 10 + 25 + 22 + 12 + 16 + 14
= 99 units

All -pairs shortest paths:

All Pairs Shortest Paths

Given a directed, connected weighted graph G(V,E), for each edge ⟨u,v⟩∈E, a weight w(u,v) is
associated with the edge. The all pairs of shortest paths problem (APSP) is to find a shortest
path from u to v for every pair of vertices u and v in V.

1 The representation of G

Page 25
 UNIT 3

2 Algorithms for the APSP problem

• Matrix Multiplication / Repeated Squaring
• The Floyd-Warshall Algorithm
• Transitive Closure of a Graph

3 Matrix Multiplication Algorithm

The algorithm is based on dynamic programming, in which each major loop will invoke an
operation that is very similar to matrix multiplication. Following the DP strategy, the structure
of this problem is, for any two vertices u and v,

1. if u=v, then the shortest path p from u to v is 0.

2. otherwise, decompose p into u→x→v, where p' is a path from u to x and contains at
most k edges and it is the shortest path from u to x.

A recursive solution for the APSP problem is defined. Let dij(k) be the minimum weight of any
path from i to j that contains at most k edges.

1. If k=0, then

SPECIAL-MATRIX-MULTIPLY (A,B)
1 n ← [A]
2 C ← new n×n matrix
3 for i ← 1 to n
4 do for j ← 1 to n
5 do cij ← ∞
6 for k ← 1 to n
7 do cij ← min(cij,aik+bkj)
. /* Here's where this algorithm */ Page 26
 UNIT 3

. /* differs from MATRIX-MULTIPLY. */

8 return C
The optimal solution can be computed by calling SPECIAL-MATRIX-MULTIPLY (D(k),W) for 1≤k≤n-2.
We only need to run to n-2 because that will give us D(n-1) giving us all the shortest path lengths
of at most n-1 edges (you only need n-1 edges to connect n vertices). Since SPECIAL-MATRIX-
MULTIPLY is called n-2 times, the total running time is O(n4).

3.1 The repeated squaring method

Since each D(k) matrix contains the shortest paths of at most k edges, and W really is D(1), all we
were doing in the earlier solution was going: "Given the shortests paths of at most length k, and
the shortests paths of at most length 1, what is the shortests paths of at most length k+1?"
This situation is ripe for improvement. The repeated squaring method rephrases the question to:
"Given the shortests paths of at most length k, what is the shortests paths of at most length k+k?"
The correctness of this approach lies in the observation that the shortests paths of at
most m edges is the same as the shortest paths of at most n-1 edges for all m>n-1. Thus:

Using repeated squaring, we only need to run SPECIAL-MATRIX-MULTIPLY ⌈log(n-1)⌉ times. Hence the
running time of the improved matrix multiplication is
ALL-PAIRS-SHORTEST-PATHS (W)

3.2 Repeat Squaring Example

Page 27
 UNIT 3

Figure 1: Example graph for the Repeated Squaring method.

Take the graph in Figure 1 for example. The weights for this graph in matrix form...

D(4) contains the all-pairs shortest paths. It is interesting to note that at D(2), the shortest path
from 2 to 1 is 9 using the path ⟨2,3,1⟩. Since the final solution ( D(4)) allows for up to 4 edges to
be used, a shorter path ⟨2,3,4,1⟩ was found with a weight of 6.

https://users.cecs.anu.edu.au/~Alistair.Rendell/Teaching/apac_comp3600/module4/all_pairs_shortest_paths.
xhtml

Floyd Warshall Algorithm-

• Floyd Warshall Algorithm is a famous algorithm.

• It is used to solve All Pairs Shortest Path Problem.
• It computes the shortest path between every pair of vertices of the given graph.
• Floyd Warshall Algorithm is an example of dynamic programming approach.
Advantages-
Floyd Warshall Algorithm has the following main advantages-
• It is extremely simple.
• It is easy to implement.
Algorithm- Floyd Warshall Algorithm is as shown below-
Time Complexity-
• Floyd Warshall Algorithm consists of three loops over all the nodes.
• The inner most loop consists of only constant complexity operations.
Page 28
 UNIT 3

• Hence, the asymptotic complexity of Floyd Warshall algorithm is O(n3).

• Here, n is the number of nodes in the given graph.
When Floyd Warshall Algorithm Is Used?
• Floyd Warshall Algorithm is best suited for dense graphs.
• This is because its complexity depends only on the number of vertices in the given graph.
• For sparse graphs, Johnson’s Algorithm is more suitable.
PRACTICE PROBLEM BASED ON FLOYD WARSHALL ALGORITHM
Consider the following directed weighted graph-

Using Floyd Warshall Algorithm, find the shortest path distance between every pair of vertices.
Solution-
Step-01:
• Remove all the self loops and parallel edges (keeping the lowest weight edge) from the graph.
• In the given graph, there are neither self edges nor parallel edges.
Step-02:
• Write the initial distance matrix.
• It represents the distance between every pair of vertices in the form of given weights.
• For diagonal elements (representing self-loops), distance value = 0.
• For vertices having a direct edge between them, distance value = weight of that edge.
• For vertices having no direct edge between them, distance value = ∞.
Initial distance matrix for the given graph is-

Step-03:
Using Floyd Warshall Algorithm, write the following 4 matrices-

Page 29
 UNIT 3

The last matrix D4 represents the shortest path distance between every pair of vertices.
Remember-
• In the above problem, there are 4 vertices in the given graph.
• So, there will be total 4 matrices of order 4 x 4 in the solution excluding the initial distance matrix.
• Diagonal elements of each matrix will always be 0.

Page 30
 UNIT 3

Single-Source Shortest Paths:

The Single-Source Shortest Path (SSSP) problem consists of finding the
shortest paths between a given vertex v and all other vertices in the graph.
Algorithms such as Breadth-First-Search (BFS) for unweighted graphs or
Dijkstra [1] solve this problem.

Bellman–Ford Algorithm
•
Imagine you have a map with different cities connected by roads, each road having a certain
distance. The Bellman–Ford algorithm is like a guide that helps you find the shortest path from
one city to all other cities, even if some roads have negative lengths. It’s like a GPS for computers,
useful for figuring out the quickest way to get from one point to another in a network. In this
article, we’ll take a closer look at how this algorithm works and why it’s so handy in solving
everyday problems.

Table of Content
• Bellman-Ford Algorithm
• The idea behind Bellman Ford Algorithm
• Principle of Relaxation of Edges for Bellman-Ford
• Why Relaxing Edges N-1 times, gives us Single Source Shortest Path?
• Why Does the Reduction of Distance in the N’th Relaxation Indicates the Existence of
a Negative Cycle?
• Working of Bellman-Ford Algorithm to Detect the Negative cycle in the graph
• Algorithm to Find Negative Cycle in a Directed Weighted Graph Using Bellman-Ford
• Handling Disconnected Graphs in the Algorithm
• Complexity Analysis of Bellman-Ford Algorithm
• Bellman Ford’s Algorithm Applications
• Drawback of Bellman Ford’s Algorithm
Page 31
 UNIT 3

2. Bellman-Ford Algorithm
Bellman-Ford is a single source shortest path algorithm that determines the shortest path
between a given source vertex and every other vertex in a graph. This algorithm can be used on
both weighted and unweighted graphs.
Recommended Problem
Distance from the Source (Bellman-Ford Algorithm)
A Bellman-Ford algorithm is also guaranteed to find the shortest path in a graph, similar
to Dijkstra’s algorithm. Although Bellman-Ford is slower than Dijkstra’s algorithm, it is
capable of handling graphs with negative edge weights, which makes it more versatile. The
shortest path cannot be found if there exists a negative cycle in the graph. If we continue to go
around the negative cycle an infinite number of times, then the cost of the path will continue to
decrease (even though the length of the path is increasing). As a result, Bellman-Ford is also
capable of detecting negative cycles, which is an important feature.
3. The idea behind Bellman Ford Algorithm:
The Bellman-Ford algorithm’s primary principle is that it starts with a single source and
calculates the distance to each node. The distance is initially unknown and assumed to be infinite,
but as time goes on, the algorithm relaxes those paths by identifying a few shorter paths. Hence
it is said that Bellman-Ford is based on “Principle of Relaxation“.
4. Principle of Relaxation of Edges for Bellman-Ford:
• It states that for the graph having N vertices, all the edges should be relaxed N-1 times
to compute the single source shortest path.
• In order to detect whether a negative cycle exists or not, relax all the edge one more
time and if the shortest distance for any node reduces then we can say that a negative
cycle exists. In short if we relax the edges N times, and there is any change in the
shortest distance of any node between the N-1th and Nth relaxation than a negative
cycle exists, otherwise not exist.
5. Why Relaxing Edges N-1 times, gives us Single Source Shortest Path?
In the worst-case scenario, a shortest path between two vertices can have at most N-1 edges,
where N is the number of vertices. This is because a simple path in a graph with N vertices can
have at most N-1 edges, as it’s impossible to form a closed loop without revisiting a vertex.
By relaxing edges N-1 times, the Bellman-Ford algorithm ensures that the distance estimates for
all vertices have been updated to their optimal values, assuming the graph doesn’t contain any
negative-weight cycles reachable from the source vertex. If a graph contains a negative-weight
cycle reachable from the source vertex, the algorithm can detect it after N-1 iterations, since the
negative cycle disrupts the shortest path lengths.
In summary, relaxing edges N-1 times in the Bellman-Ford algorithm guarantees that the
algorithm has explored all possible paths of length up to N-1, which is the maximum possible
length of a shortest path in a graph with N vertices. This allows the algorithm to correctly calculate
the shortest paths from the source vertex to all other vertices, given that there are no negative-
weight cycles.
6. Why Does the Reduction of Distance in the N’th Relaxation Indicates the
Existence of a Negative Cycle?
As previously discussed, achieving the single source shortest paths to all other nodes takes N-
1 relaxations. If the N’th relaxation further reduces the shortest distance for any node, it implies
that a certain edge with negative weight has been traversed once more. It is important to note that
during the N-1 relaxations, we presumed that each vertex is traversed only once. However, the
reduction of distance during the N’th relaxation indicates revisiting a vertex.
Page 32
 UNIT 3

7. Working of Bellman-Ford Algorithm to Detect the Negative cycle in the

graph:
Let’s suppose we have a graph which is given below and we want to find whether there exists a
negative cycle or not using Bellman-Ford.

Initial Graph

Step 1: Initialize a distance array Dist[] to store the shortest distance for each vertex from the
source vertex. Initially distance of source will be 0 and Distance of other vertices will be
INFINITY.

Initialize a distance array

Step 2: Start relaxing the edges, during 1st Relaxation:

• Current Distance of B > (Distance of A) + (Weight of A to B) i.e. Infinity > 0 + 5
o Therefore, Dist[B] = 5

Page 33
 UNIT 3

1st Relaxation

Step 3: During 2nd Relaxation:

• Current Distance of D > (Distance of B) + (Weight of B to D) i.e. Infinity > 5 + 2
o Dist[D] = 7
• Current Distance of C > (Distance of B) + (Weight of B to C) i.e. Infinity > 5 + 1
o Dist[C] = 6

2nd Relaxation

Step 4: During 3rd Relaxation:

• Current Distance of F > (Distance of D ) + (Weight of D to F) i.e. Infinity > 7 + 2
o Dist[F] = 9
• Current Distance of E > (Distance of C ) + (Weight of C to E) i.e. Infinity > 6 + 1
o Dist[E] = 7

Page 34
 UNIT 3

3rd Relaxation

Step 5: During 4th Relaxation:

• Current Distance of D > (Distance of E) + (Weight of E to D) i.e. 7 > 7 + (-1)
o Dist[D] = 6
• Current Distance of E > (Distance of F ) + (Weight of F to E) i.e. 7 > 9 + (-3)
o Dist[E] = 6

4th Relaxation

Step 6: During 5th Relaxation:

• Current Distance of F > (Distance of D) + (Weight of D to F) i.e. 9 > 6 + 2
o Dist[F] = 8
• Current Distance of D > (Distance of E ) + (Weight of E to D) i.e. 6 > 6 + (-1)
o Dist[D] = 5
• Since the graph h 6 vertices, So during the 5th relaxation the shortest distance for all
the vertices should have been calculated.
Page 35
 UNIT 3

5th Relaxation

Step 7: Now the final relaxation i.e. the 6th relaxation should indicate the presence of negative
cycle if there is any changes in the distance array of 5th relaxation.
During the 6th relaxation, following changes can be seen:
• Current Distance of E > (Distance of F) + (Weight of F to E) i.e. 6 > 8 + (-3)
o Dist[E]=5
• Current Distance of F > (Distance of D ) + (Weight of D to F) i.e. 8 > 5 + 2
o Dist[F]=7
Since, we observer changes in the Distance array Hence ,we can conclude the presence of a
negative cycle in the graph.

6th Relaxation

Result: A negative cycle (D->F->E) exists in the graph.

8. Algorithm to Find Negative Cycle in a Directed Weighted Graph Using
Bellman-Ford:
• Initialize distance array dist[] for each vertex ‘v‘ as dist[v] = INFINITY.
• Assume any vertex (let’s say ‘0’) as source and assign dist = 0.
• Relax all the edges(u,v,weight) N-1 times as per the below condition:
o dist[v] = minimum(dist[v], distance[u] + weight)
• Now, Relax all the edges one more time i.e. the Nth time and based on the below two
cases we can detect the negative cycle:
o Case 1 (Negative cycle exists): For any edge(u, v, weight), if dist[u] +
weight < dist[v]
o Case 2 (No Negative cycle) : case 1 fails for all the edges.
9. Handling Disconnected Graphs in the Algorithm:
The above algorithm and program might not work if the given graph is disconnected. It works
when all vertices are reachable from source vertex 0.
To handle disconnected graphs, we can repeat the above algorithm for vertices having distance
= INFINITY, or simply for the vertices that are not visited.
Below is the implementation of the above approach:
Page 36
 UNIT 3

C++JavaPythonC#JavaScript
// A Java program for Bellman-Ford's single source shortest
// path algorithm.

import java.io.*;
import java.lang.*;
import java.util.*;

// A class to represent a connected, directed and weighted

// graph
class Graph {

// A class to represent a weighted edge in graph

class Edge {
int src, dest, weight;
Edge() { src = dest = weight = 0; }
};

int V, E;
Edge edge[];

// Creates a graph with V vertices and E edges

Graph(int v, int e)
{
V = v;
E = e;
edge = new Edge[e];
for (int i = 0; i < e; ++i)
edge[i] = new Edge();
}

// The main function that finds shortest distances from

// src to all other vertices using Bellman-Ford
// algorithm. The function also detects negative weight
// cycle
void BellmanFord(Graph graph, int src)
{
int V = graph.V, E = graph.E;
int dist[] = new int[V];

// Step 1: Initialize distances from src to all

// other vertices as INFINITE
for (int i = 0; i < V; ++i)
dist[i] = Integer.MAX_VALUE;
dist[src] = 0;

// Step 2: Relax all edges |V| - 1 times. A simple

// shortest path from src to any other vertex can
// have at-most |V| - 1 edges
for (int i = 1; i < V; ++i) {
for (int j = 0; j < E; ++j) {
int u = graph.edge[j].src;
int v = graph.edge[j].dest;
int weight = graph.edge[j].weight;
if (dist[u] != Integer.MAX_VALUE
&& dist[u] + weight < dist[v])
dist[v] = dist[u] + weight;
}
Page 37
 UNIT 3

// Step 3: check for negative-weight cycles. The

// above step guarantees shortest distances if graph
// doesn't contain negative weight cycle. If we get
// a shorter path, then there is a cycle.
for (int j = 0; j < E; ++j) {
int u = graph.edge[j].src;
int v = graph.edge[j].dest;
int weight = graph.edge[j].weight;
if (dist[u] != Integer.MAX_VALUE
&& dist[u] + weight < dist[v]) {
System.out.println(
"Graph contains negative weight cycle");
return;
}
}
printArr(dist, V);
}

// A utility function used to print the solution

void printArr(int dist[], int V)
{
System.out.println("Vertex Distance from Source");
for (int i = 0; i < V; ++i)
System.out.println(i + "\t\t" + dist[i]);
}

// Driver's code
public static void main(String[] args)
{
int V = 5; // Number of vertices in graph
int E = 8; // Number of edges in graph

Graph graph = new Graph(V, E);

// add edge 0-1 (or A-B in above figure)

graph.edge[0].src = 0;
graph.edge[0].dest = 1;
graph.edge[0].weight = -1;

// add edge 0-2 (or A-C in above figure)

graph.edge[1].src = 0;
graph.edge[1].dest = 2;
graph.edge[1].weight = 4;

// add edge 1-2 (or B-C in above figure)

graph.edge[2].src = 1;
graph.edge[2].dest = 2;
graph.edge[2].weight = 3;

// add edge 1-3 (or B-D in above figure)

graph.edge[3].src = 1;
graph.edge[3].dest = 3;
graph.edge[3].weight = 2;

// add edge 1-4 (or B-E in above figure)

graph.edge[4].src = 1;
Page 38
 UNIT 3

graph.edge[4].dest = 4;
graph.edge[4].weight = 2;

// add edge 3-2 (or D-C in above figure)

graph.edge[5].src = 3;
graph.edge[5].dest = 2;
graph.edge[5].weight = 5;

// add edge 3-1 (or D-B in above figure)

graph.edge[6].src = 3;
graph.edge[6].dest = 1;
graph.edge[6].weight = 1;

// add edge 4-3 (or E-D in above figure)

graph.edge[7].src = 4;
graph.edge[7].dest = 3;
graph.edge[7].weight = -3;

// Function call
graph.BellmanFord(graph, 0);
}
}
// Contributed by Aakash Hasija

Output
Vertex Distance from Source
0 0
1 -1
2 2
3 -2
4 1

10. Complexity Analysis of Bellman-Ford Algorithm:

• Time Complexity when graph is connected:
o Best Case: O(E), when distance array after 1st and 2nd relaxation are same ,
we can simply stop further processing
o Average Case: O(V*E)
o Worst Case: O(V*E)
• Time Complexity when graph is disconnected:
o All the cases: O(E*(V^2))
• Auxiliary Space: O(V), where V is the number of vertices in the graph.
11. Bellman Ford’s Algorithm Applications:
• Network Routing: Bellman-Ford is used in computer networking to find the shortest
paths in routing tables, helping data packets navigate efficiently across networks.
• GPS Navigation: GPS devices use Bellman-Ford to calculate the shortest or fastest
routes between locations, aiding navigation apps and devices.
• Transportation and Logistics: Bellman-Ford’s algorithm can be applied to
determine the optimal paths for vehicles in transportation and logistics, minimizing
fuel consumption and travel time.

Page 39
 UNIT 3

• Game Development: Bellman-Ford can be used to model movement and navigation

within virtual worlds in game development, where different paths may have varying
costs or obstacles.
• Robotics and Autonomous Vehicles: The algorithm aids in path planning for robots
or autonomous vehicles, considering obstacles, terrain, and energy consumption.
12. Drawback of Bellman Ford’s Algorithm:
• Bellman-Ford algorithm will fail if the graph contains any negative edge cycle.

4. What is Dijkstra’s Algorithm? | Introduction to

Dijkstra’s Shortest Path Algorithm
Last Updated : 09 May, 2024

•
In this article, we will be discussing one of the most commonly known shortest-path algorithms
i.e. Dijkstra’s Shortest Path Algorithm which was developed by Dutch computer scientist
Edsger W. Dijkstra in 1956. Moreover, we will do a complexity analysis for this algorithm and
also see how it differs from other shortest-path algorithms.
Table of Content
• Dijkstra’s Algorithm
• Need for Dijkstra’s Algorithm (Purpose and Use-Cases)
• Can Dijkstra’s Algorithm work on both Directed and Undirected graphs?
• Algorithm for Dijkstra’s Algorithm
• How does Dijkstra’s Algorithm works?
• Pseudo Code for Dijkstra’s Algorithm
• Implemention of Dijkstra’s Algorithm:
• Dijkstra’s Algorithms vs Bellman-Ford Algorithm
• Dijkstra’s Algorithm vs Floyd-Warshall Algorithm
• Dijkstra’s Algorithm vs A* Algorithm
• Practice Problems on Dijkstra’s Algorithm
• Conclusion:

Page 40
 UNIT 3

1. Dijkstra’s Algorithm:
Dijkstra’s algorithm is a popular algorithms for solving many single-source shortest path
problems having non-negative edge weight in the graphs i.e., it is to find the shortest distance
between two vertices on a graph. It was conceived by Dutch computer scientist Edsger W.
Dijkstra in 1956.
The algorithm maintains a set of visited vertices and a set of unvisited vertices. It starts at the
source vertex and iteratively selects the unvisited vertex with the smallest tentative distance
from the source. It then visits the neighbors of this vertex and updates their tentative distances if
a shorter path is found. This process continues until the destination vertex is reached, or all
reachable vertices have been visited.
2. Need for Dijkstra’s Algorithm (Purpose and Use-Cases)
The need for Dijkstra’s algorithm arises in many applications where finding the shortest path
between two points is crucial.
For example, It can be used in the routing protocols for computer networks and also used by
map systems to find the shortest path between starting point and the Destination (as explained
in How does Google Maps work?)
3. Can Dijkstra’s Algorithm work on both Directed and Undirected graphs?
Yes, Dijkstra’s algorithm can work on both directed graphs and undirected graphs as this
algorithm is designed to work on any type of graph as long as it meets the requirements of
having non-negative edge weights and being connected.
• In a directed graph, each edge has a direction, indicating the direction of travel
between the vertices connected by the edge. In this case, the algorithm follows the
direction of the edges when searching for the shortest path.
• In an undirected graph, the edges have no direction, and the algorithm can traverse
both forward and backward along the edges when searching for the shortest path.
4. Algorithm for Dijkstra’s Algorithm:
1. Mark the source node with a current distance of 0 and the rest with infinity.
2. Set the non-visited node with the smallest current distance as the current node.
3. For each neighbor, N of the current node adds the current distance of the adjacent
node with the weight of the edge connecting 0->1. If it is smaller than the current
distance of Node, set it as the new current distance of N.
4. Mark the current node 1 as visited.
5. Go to step 2 if there are any nodes are unvisited.
5. How does Dijkstra’s Algorithm works?
Let’s see how Dijkstra’s Algorithm works with an example given below:
Dijkstra’s Algorithm will generate the shortest path from Node 0 to all other Nodes in the graph.
Consider the below graph:

Page 41
 UNIT 3

Dijkstra’s Algorithm

The algorithm will generate the shortest path from node 0 to all the other nodes in the graph.
For this graph, we will assume that the weight of the edges represents the distance between
two nodes.
As, we can see we have the shortest path from,
Node 0 to Node 1, from
Node 0 to Node 2, from
Node 0 to Node 3, from
Node 0 to Node 4, from
Node 0 to Node 6.
Initially we have a set of resources given below :
• The Distance from the source node to itself is 0. In this example the source node is 0.
• The distance from the source node to all other node is unknown so we mark all of
them as infinity.
Example: 0 -> 0, 1-> ∞,2-> ∞,3-> ∞,4-> ∞,5-> ∞,6-> ∞.
• we’ll also have an array of unvisited elements that will keep track of unvisited or
unmarked Nodes.
• Algorithm will complete when all the nodes marked as visited and the distance
between them added to the path. Unvisited Nodes:- 0 1 2 3 4 5 6.
Step 1: Start from Node 0 and mark Node as visited as you can check in below image visited
Node is marked red.

Page 42
 UNIT 3

Dijkstra’s Algorithm

Step 2: Check for adjacent Nodes, Now we have to choices (Either choose Node1 with distance
2 or either choose Node 2 with distance 6 ) and choose Node with minimum distance. In this
step Node 1 is Minimum distance adjacent Node, so marked it as visited and add up the
distance.
Distance: Node 0 -> Node 1 = 2

Dijkstra’s Algorithm

Step 3: Then Move Forward and check for adjacent Node which is Node 3, so marked it as
visited and add up the distance, Now the distance will be:
Distance: Node 0 -> Node 1 -> Node 3 = 2 + 5 = 7

Page 43
 UNIT 3

Dijkstra’s Algorithm

Step 4: Again we have two choices for adjacent Nodes (Either we can choose Node 4 with
distance 10 or either we can choose Node 5 with distance 15) so choose Node with minimum
distance. In this step Node 4 is Minimum distance adjacent Node, so marked it as visited and
add up the distance.
Distance: Node 0 -> Node 1 -> Node 3 -> Node 4 = 2 + 5 + 10 = 17

Dijkstra’s Algorithm

Step 5: Again, Move Forward and check for adjacent Node which is Node 6, so marked it as
visited and add up the distance, Now the distance will be:
Distance: Node 0 -> Node 1 -> Node 3 -> Node 4 -> Node 6 = 2 + 5 + 10 + 2 = 19

Page 44
 UNIT 3

Dijkstra’s Algorithm

So, the Shortest Distance from the Source Vertex is 19 which is optimal one
6. Pseudo Code for Dijkstra’s Algorithm
function Dijkstra(Graph, source):
// Initialize distances to all nodes as infinity, and to the source node as 0.
distances = map(all nodes -> infinity)
distances = 0
// Initialize an empty set of visited nodes and a priority queue to keep track of the nodes to
visit.
visited = empty set
queue = new PriorityQueue()
queue.enqueue(source, 0)
// Loop until all nodes have been visited.
while queue is not empty:
// Dequeue the node with the smallest distance from the priority queue.
current = queue.dequeue()
// If the node has already been visited, skip it.
if current in visited:
continue
// Mark the node as visited.
visited.add(current)
// Check all neighboring nodes to see if their distances need to be updated.
for neighbor in Graph.neighbors(current):
// Calculate the tentative distance to the neighbor through the current node.
tentative_distance = distances[current] + Graph.distance(current, neighbor)
// If the tentative distance is smaller than the current distance to the neighbor, update the
distance.
if tentative_distance < distances[neighbor]:
distances[neighbor] = tentative_distance
// Enqueue the neighbor with its new distance to be considered for visitation in the
future.
queue.enqueue(neighbor, distances[neighbor]) Page 45
 UNIT 3

// Return the calculated distances from the source to all other nodes in the graph.
return distances

7. Implemention of Dijkstra’s Algorithm:

There are several ways to Implement Dijkstra’s algorithm, but the most common ones are:
1. Priority Queue (Heap-based Implementation):
2. Array-based Implementation:
1. 1. Dijkstra’s Shortest Path Algorithm using priority_queue (Heap)
In this Implementation, we are given a graph and a source vertex in the graph, finding the
shortest paths from the source to all vertices in the given graph.
Example:
Input: Source = 0

Example

Output: Vertex
Vertex Distance from Source
0 -> 0
1 -> 2
2 -> 6
3 -> 7
4 -> 17
5 -> 22
6 -> 19
Below is the algorithm based on the above idea:
• Initialize the distance values and priority queue.
• Insert the source node into the priority queue with distance 0.
• While the priority queue is not empty:
o Extract the node with the minimum distance from the priority queue.
o Update the distances of its neighbors if a shorter path is found.
o Insert updated neighbors into the priority queue.
Below is the C++ Implementation of the above approach:
C++JavaPythonC#JavaScript
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.PriorityQueue;

Page 46
 UNIT 3

public class DijkstraAlgoForShortestDistance {

static class Node implements Comparable<Node> {

int v;
int distance;

public Node(int v, int distance)

{
this.v = v;
this.distance = distance;
}

@Override public int compareTo(Node n)

{
if (this.distance <= n.distance) {
return -1;
}
else {
return 1;
}
}
}

static int[] dijkstra(

int V,
ArrayList<ArrayList<ArrayList<Integer> > > adj,
int S)
{
boolean[] visited = new boolean[V];
HashMap<Integer, Node> map = new HashMap<>();
PriorityQueue<Node> q = new PriorityQueue<>();

map.put(S, new Node(S, 0));

q.add(new Node(S, 0));

while (!q.isEmpty()) {
Node n = q.poll();
int v = n.v;
int distance = n.distance;
visited[v] = true;

ArrayList<ArrayList<Integer> > adjList

= adj.get(v);
for (ArrayList<Integer> adjLink : adjList) {

if (visited[adjLink.get(0)] == false) {
if (!map.containsKey(adjLink.get(0))) {
map.put(
adjLink.get(0),
new Node(v,
distance
+ adjLink.get(1)));
}
else {
Node sn = map.get(adjLink.get(0));
if (distance + adjLink.get(1)
< sn.distance) {
sn.v = v;
Page 47
 UNIT 3

sn.distance
= distance + adjLink.get(1);
}
}
q.add(new Node(adjLink.get(0),
distance
+ adjLink.get(1)));
}
}
}

int[] result = new int[V];

for (int i = 0; i < V; i++) {
result[i] = map.get(i).distance;
}

return result;
}

public static void main(String[] args)

{
ArrayList<ArrayList<ArrayList<Integer> > > adj
= new ArrayList<>();
HashMap<Integer, ArrayList<ArrayList<Integer> > >
map = new HashMap<>();

int V = 6;
int E = 5;
int[] u = { 0, 0, 1, 2, 4 };
int[] v = { 3, 5, 4, 5, 5 };
int[] w = { 9, 4, 4, 10, 3 };

for (int i = 0; i < E; i++) {

ArrayList<Integer> edge = new ArrayList<>();
edge.add(v[i]);
edge.add(w[i]);

ArrayList<ArrayList<Integer> > adjList;

if (!map.containsKey(u[i])) {
adjList = new ArrayList<>();
}
else {
adjList = map.get(u[i]);
}
adjList.add(edge);
map.put(u[i], adjList);

ArrayList<Integer> edge2 = new ArrayList<>();

edge2.add(u[i]);
edge2.add(w[i]);

ArrayList<ArrayList<Integer> > adjList2;

if (!map.containsKey(v[i])) {
adjList2 = new ArrayList<>();
}
else {
adjList2 = map.get(v[i]);
}
Page 48
 UNIT 3

adjList2.add(edge2);
map.put(v[i], adjList2);
}

for (int i = 0; i < V; i++) {

if (map.containsKey(i)) {
adj.add(map.get(i));
}
else {
adj.add(null);
}
}
int S = 1;

// Input sample
//[0 [[3, 9], [5, 4]],
// 1 [[4, 4]],
// 2 [[5, 10]],
// 3 [[0, 9]],
// 4 [[1, 4], [5, 3]],
// 5 [[0, 4], [2, 10], [4, 3]]
//]
int[] result
= DijkstraAlgoForShortestDistance.dijkstra(
V, adj, S);
System.out.println(Arrays.toString(result));
}
}
Final Answer:

Output

Complexity Analysis of Dijkstra Algorithm:

• Time complexity: O((V + E) log V), where V is the number of vertices and E is the
number of edges.
• Auxiliary Space: O(V), where V is the number of vertices and E is the number of
edges in the graph.

Page 49
 UNIT 3

2. 2. Array-based Implementation of Dijkstra’s Algorithm (Naive

Approach):
Dijkstra’s Algorithm can also be implemented using arrays without using a priority queue. This
implementation is straightforward but can be less efficient, especially on large graphs.
The algorithm proceeds as follows:
• Initialize an array to store distances from the source to each node.
• Mark all nodes as unvisited.
• Set the distance to the source node as 0 and infinity for all other nodes.
• Repeat until all nodes are visited:
o Find the unvisited node with the smallest known distance.
o For the current node, update the distances of its unvisited neighbors.
o Mark the current node as visited.
Complexity Analysis:
• Time Complexity: O(V^2) in the worst case, where V is the number of vertices. This
can be improved to O(V^2) with some optimizations.
• Auxiliary Space: O(V), where V is the number of vertices and E is the number of
edges in the graph.
8. Dijkstra’s Algorithms vs Bellman-Ford Algorithm
Dijkstra’s algorithm and Bellman-Ford algorithm are both used to find the shortest path in a
weighted graph, but they have some key differences. Here are the main differences between
Dijkstra’s algorithm and Bellman-Ford algorithm:
Feature: Dijkstra’s Bellman Ford

Bellman-Ford algorithm is
optimized for finding the
optimized for finding the
shortest path between a single
shortest path between a single
source node and all other nodes
source node and all other nodes
in a graph with non-negative
in a graph with negative edge
edge weights
Optimization weights.

Dijkstra’s algorithm uses a the Bellman-Ford algorithm

greedy approach where it relaxes all edges in each
chooses the node with the iteration, updating the distance
smallest distance and updates of each node by considering all
Relaxation its neighbors possible paths to that node

Dijkstra’s algorithm has a time

Bellman-Ford algorithm has a
complexity of O(V^2) for a
time complexity of O(VE),
dense graph and O(E log V) for
where V is the number of
a sparse graph, where V is the
vertices and E is the number of
number of vertices and E is the
edges in the graph.
Time Complexity number of edges in the graph.

Dijkstra’s algorithm does not

Bellman-Ford algorithm can
work with graphs that have
handle negative edge weights
negative edge weights, as it
and can detect negative-weight
assumes that all edge weights
cycles in the graph.
Negative Weights are non-negative.

Page 50
 UNIT 3

9. Dijkstra’s Algorithm vs Floyd-Warshall Algorithm

Dijkstra’s algorithm and Floyd-Warshall algorithm are both used to find the shortest path in a
weighted graph, but they have some key differences. Here are the main differences between
Dijkstra’s algorithm and Floyd-Warshall algorithm:
Floyd-Warshall
Feature: Dijkstra’s Algorithm

Optimized for finding the

Floyd-Warshall algorithm is
shortest path between a single
optimized for finding the
source node and all other nodes
shortest path between all pairs
in a graph with non-negative
of nodes in a graph.
Optimization edge weights

Floyd-Warshall algorithm, on
Dijkstra’s algorithm is a single-
the other hand, is an all-pairs
source shortest path algorithm
shortest path algorithm that
that uses a greedy approach
uses dynamic programming to
and calculates the shortest path
calculate the shortest path
from the source node to all
between all pairs of nodes in
other nodes in the graph.
Technique the graph.

Floyd-Warshall algorithm, on
Dijkstra’s algorithm has a time
the other hand, is an all-pairs
complexity of O(V^2) for a
shortest path algorithm that
dense graph and O(E log V) for
uses dynamic programming to
a sparse graph, where V is the
calculate the shortest path
number of vertices and E is the
between all pairs of nodes in
number of edges in the graph.
Time Complexity the graph.

Floyd-Warshall algorithm, on
Dijkstra’s algorithm does not the other hand, is an all-pairs
work with graphs that have shortest path algorithm that
negative edge weights, as it uses dynamic programming to
assumes that all edge weights calculate the shortest path
are non-negative. between all pairs of nodes in
Negative Weights the graph.

10. Dijkstra’s Algorithm vs A* Algorithm

Dijkstra’s algorithm and A* algorithm are both used to find the shortest path in a weighted
graph, but they have some key differences. Here are the main differences between Dijkstra’s
algorithm and A* algorithm:
Feature: A* Algorithm

Optimized for finding the

A* algorithm is an informed
shortest path between a single
search algorithm that uses a
source node and all other nodes
heuristic function to guide the
in a graph with non-negative
search towards the goal node.
Search Technique edge weights

Page 51
 UNIT 3

Feature: A* Algorithm

A* algorithm uses a heuristic

function that estimates the
Dijkstra’s algorithm, does not
distance from the current node
use any heuristic function and
to the goal node. This heuristic
considers all the nodes in the
function is admissible, meaning
graph.
that it never overestimates the
Heuristic Function actual distance to the goal node

Dijkstra’s algorithm has a time

complexity of O(V^2) for a The time complexity of A*
dense graph and O(E log V) for algorithm depends on the
a sparse graph, where V is the quality of the heuristic
number of vertices and E is the function.
Time Complexity number of edges in the graph.

Dijkstra’s algorithm is used in . A* algorithm is commonly

many applications such as used in pathfinding and graph
routing algorithms, GPS traversal problems, such as
navigation systems, and video games, robotics, and
Application network analysis planning algorithms.

11. Practice Problems on Dijkstra’s Algorithm:

1. Dijkstra’s shortest path algorithm | Greedy Algo-7
2. Dijkstra’s Algorithm for Adjacency List Representation | Greedy Algo-8
3. Dijkstra’s Algorithm – Priority Queue and Array Implementation
4. Dijkstra’s shortest path algorithm using set in STL
5. Dijkstra’s shortest path algorithm using STL in C++
6. Dijkstra’s Shortest Path Algorithm using priority_queue of STL
7. Dijkstra’s shortest path algorithm using matrix in C++
8. Dijkstra’s Algorithm for Single Source Shortest Path in a DAG
9. Dijkstra’s Algorithm using Fibonacci Heap
10. Dijkstra’s shortest path algorithm for directed graph with negative weights
11. Printing Paths in Dijkstra’s Shortest Path Algorithm
12. Dijkstra’s shortest path algorithm with priority queue in Java
13. Dijkstra’s shortest path algorithm with adjacency list in Java
14. Dijkstra’s shortest path algorithm using adjacency matrix in Java
15. Dijkstra’s Algorithm using Adjacency List in Python
16. Dijkstra’s Algorithm using PriorityQueue in Python
17. Dijkstra’s Algorithm using heapq module in Python
18. Dijkstra’s Algorithm using dictionary and priority queue in Python
12. Conclusion:
Overall, Dijkstra’s Algorithm is a simple and efficient way to find the shortest path in a graph
with non-negative edge weights. However, it may not work well with graphs that have negative
edge weights or cycles. In such cases, more advanced algorithms such as the Bellman-Ford
algorithm or the Floyd-Warshall algorithm may be used.

Page 52
 Red black tree is
Red Black Tree
we can
another variant of binary search tree in
define a red black tree which every node is colored either red
as follows: or black
Red black tree is a
binary search tree in which every node is
colored either red or black.
A red-black tree's node structure
would be:
struct t_red black_node
enum red, black colour;
void *item;
struct t_red_black_node *left,

*right,
*parent;
In redblack tree the color of node
is decided based on the
tree has the
following properties of red black
tree,Every red black
properties:
1. Red black tree must be a
binary search tree.
2.The root node must be
colored black.
3. The children of red color node must be colored black. There should not be two consecutive
4. In all the
red nodes.
paths of the tree there should be same number of
black color nodes.
5. Every new node must be
inserted with red color.
6. Every leaf( i.e null node) must be colored black.
Example:
Following is a red black tree which is created
by inserting number from 1 to 9.

The above tree is a red black tree where every node is

satistying all the properties of red black tree.
 Insertion into red black tree:
In red black tree,
a
every new node must be inserted with the
black tree is similar to insertion color red. The insertion
operation in binary search tree. operation in red
After every insertion But it is inserted with a color
operation, we need to check all the property.
are satisfied then we
go to next operation otherwise we
properties of red black tree. If all the
properties
black tree. perform the following operation to make it red
The operations are

1. Recolor

2. Rotation

3. Rotation followed by recolor.

The insertion operation in red black tree is
performed using the following steps:
Step 1: Check whether tree is empty.

Step2: If tree is empty when insert the newnode as root node with color black
operation. an exit from the

Step3: If tree is not empty then insert the newnode as leaf node with color red.
Step4:If the parent of newnode is black then exit from the operation.

Step5: If the parent of newnode is r red then change the color of parent node's
sibling of newnode.
Step6:If it is colored black or null then make suitable rotation and recolor it.
Step7: If it is colored red then perform recolor.

Repeat the same until tree becomes red black tree.

Example:
Create a red black tree by inserting following sequence ofnumber
8, 18, 5, 15, 17, 25, 40, and 80

Insert (8)

Tree is empty. So insert newnode as root node with black color

8
Insert (18)
Tree is not
empty. So insert
newnode with red color.

18

Insert (5)

Tree is not empty. So insert newnode with red color.

18

Insert (15)

Tree is not empty. So insert newnode with red color.

8
Here there are two consecutive red nodes 18 and 15.
The newnode's parent sibling color is red and
parent's parent is root node. So we use recolor to
18 make it red black tree.

15

After recolor

After recolor operation, the trees satisfying

18
5
all red black tree properties.

15
 Insert (17)

The tree is not

empty. So insert newnode with red color.

5 18 Here there two consecutive

red nodes 15 and
17.The newnode's
parent sibling is null. So we
need rotation. Here we need
LR rotation and
15 recolor.

After left rotation 17

8
8
After right rotation and recolor

5 18

5 17

15 18
15

Insert (25)

Tree is not empty. So insert newnode with red color.

There are two consecutive red nodes 18 and

5 17 25.The newnode's parent sibling color is red
and parent's parent is not root node. So we
use recolor and recheck

15 18

25
 After recolor

5 After recolor operation, the tree is

satisfying all red black tree
properties.

15
18

Insert( 40)
Bloue

Tree is not empty. So insert newnode with red

color.

Here there are two consecutive red nodes

17
25 and 40. The newnode's parent sibling is
null. So we need rotation and recolor.
Here we use LL rotation and recheck.
15 18

After LL rotation and recolor 40

After LL rotation and recolor operation,

17 the tree satisfying all red black tree
properties.

15 25

40
18
 Insert(80)
Tree is not
empty. So insert newnode with red color.

17
There are twoconsecutive red nodes 40 and
80. The newnodes
parent sibling color is red
and parent's parent is not root
node. So we
use recolor and recheck.
15
25

18
40

After recolor
80

After recolor again there are two consecutive

17
red nodes 17 and 25. The newnode's parent
sibling color is black. So we need rotation.
We use left rotation And recolor.
15 25

18
40

80

After left rotation And recolor

 17

25

5 15 18
40

80

Finally above tree is satisfying all the properties of red black tree and it is a perfect red black tree.

Deletion operation in red black tree

The deletion operation in red

black tree is similar to deletion operation in BST. But after every deletion
operation we need to check with the red black tree properties. If any of the properties violated then
make suitable operations like recolor, rotation and rotation followed
by recolor to make it red black
tree.

Operations on Red Black Tree in details:

Rotations:

A rotation is a iocal operation in a search tree that preserves in-order traversal key ordering.

Note that in both trees, an in-order traversal yields

AKByC
The left_rotate operation may be encoded:

l e f t r o t a t e ( Tree T, node 1
node yi
-r
ight
/* Turn y's left sub-tree into x's right sub-tree /
->right= y-1eft;
i f y-2left NULL
y-left->parent Xj
wasx's parent /
/* y's new parent
Y-parent ->parenti
 * Set the parent to point to y instead of x*
/*First see whether we' re at the root *
if
X->parent= NULL )T->root Yi
else
if X== (x->parent) ->left)
/*Xwas on the left of its
X-parent->left = y;
parent */
else
*Xmust have been on the right */
x->parent->right = Y
/* Finally, put
y->left = X;
x on y's left */
X->parent= Y;

Insertion:

Insertion is somewhat complex and involves a

number of cases. Note that we start by
node, x, in the tree just as we would for any other inserting the new
binary tree, using the tree_insert function. This
new node is labelled
red, and possibly destroys the red-black property. The main
tree, restoring the red-black
loop moves up the
property.
Algorithm to Insert a New Node

Following steps are followed for inserting a new element into a red-black tree:
1. The newNode be:

20
2. Let y be the leaf (ie. NIL) and x be the root of the tree. The new node is inserted in the
following tree,

43 53
11 2
nil nil nil nil nil

nil nil nil nil

3.Check if
the tree is empty (ie. whether is
it black. x
NIL). If yes, insert newNode as a root node and color
4. Else, repeat steps following
steps until leaf (NIL) is reached.
a.
Compare newKey with rootKey.
b. lf
newKey is greater than rootKey, traverse
C. Else traverse through the left through the right subtree.
subtree.

2033
20»13 33
20 21
2021 13 63
11 2 6
nil nil nil nil nil
nil nil nil nil
5. Assign the parent of the leaf as parent of newNode.
6. If lea fKey is greater than newKey, make newNode as ri ghtChi ld.
7. Else, make newNode as leftChil1d.

20
8. Assign NULL to the left and right Child of newNode.
9. Assign RED color to newNode.

20
nil nil
 10. Call InsertFix-algorithm to maintain the property of red-black tree if violated.

Why newly inserted nodes are always red in a red-black tree?

This is because inserting a red node does not violate the depth property of a red-black tree.
Ifyou attach a red node to a red node, then the rule is violated but it is easier to fix this problem
than the problem introduced by violating the depth
property.

Algorithm to Maintain Red-Black Property After Insertion

This algorithm is used for maintaining the property of a red-black tree if insertion of a newNode
violates this property.

1. Do the following until the

parent of newNode p is RED.
2. If p is the left child of
grandParent gP of newNode, do the following
Case-l:
a. If the color of the right child of gP of newNode is
RED, set the color of both the children
of gP as BLACK and the color of gP as RED.

Case-I(a)

Change 2
the color

current
newNode
 b. Assign gP to newNode.

Case-I(b)

reassigning gP
as newNode

81
20
Case-ll:
C.(Before moving on to this step, while loop is checked. If conditions are not satisfied, it
the loop is broken.)
Else if newNode is the right child of p then, assign p to newNode.

Case-l(a)
assign p
as newNode 33
13
1 2
current

13 81 newNode
 d. Left-Rotate newNode.

Case-(b)

13 Left-Rotate
21)
1 21 43 31
15 31 11 15
Case-ll1:
e. (Before moving on to this step, while loop is checked. If conditions are not satisfied, it
the loop is broken.)
Set color of p as BLACK and color of gP as RED.

Case-l(a)
Change
the color 33
2 63 21 63
81 13 31
 f. Right-Rotate gP.

Case-1ll(b)
Right-Rotate
83
2 13 33

3 Else, do the following.

a. If the color of the left child of gP of z is RED, set the color of both the children of gP as
BLACK and the color of gP as RED.
b. Assign gP to newNode.
C. Else if newNode is the left child of p then, assign p to newNode and Right-Rotate
newNode.
d. Set color of p as BLACK and color of gP as RED.
e. Left-Rotate gP.
4. (This step is perfomed after coming out of the while loop.)
Set the root of the tree as BLACK.

Set root
color black 21
13 33

The final tree look like this:

 Final Tree

21
13 83
15 31 53
nil nil nil
20 nil nil
41) 661
nil nil nil nil nil nil

The insertion operation is encoded as:

rb insert ( Tree T', node x )

* I n s e r t in the tree in the usual way *
tree_insert ( T, X )
*Now restore the red-black property */
X->colour = red;

while ( (x != T->root) && (x->parent - >colour = red))

1 X->parent X->parent->parent->left )
* If x's parent is a left, Y is x's right 'uncle' */
y = x->parent->parent->right;

if y->colour == red )
/* case 1 - change the Colours */

X-parent->colour = black

Y->colour = black;

X->parent->parent->colour = red;

/* Move x up the tree */

X X->parent->parent;

else f
/*Y is a black node */
if (X X->parent->right)
/* a n d x i s to the right */
c a s e 2 - move x up and rotate */
x X->parent;
)
left _rotate ( T,
x
 * case 3 */
X->parent->colour = black;

X->parent->parent->colour = red;

right_rotate ( T, x->parent ->parent )

else
/* repeat the "if" part with right and left
exchanged */

/* Colour the root black */

T->root->colour = black;

Examination of the code reveals only one loop. In that loop, the node at the root of the
sub-tree whose red-black property we are trying to restore, x, may be moved up the tree at least
one level in each iteration of the loop. Since the tree originally has O(log n) height, there are
Oqlog n) iterations. The tree_insert routine also has O(log n) complexity, so overall the rb_insert
routine also has O(log n) complexity.

Red-black trees:
Trees which remain balanced and thus guarantee O(logn) search times - in a dynamic
environment. Or more importantly, since any tree can be re-balanced but at considerable
cost can be re-balanced in O(logn) time.

Time complexity in big Onotation

Algorithm Average Worst case
Space Om) On)
Search O(log n)lo(log n)
Insert Olog n) O(log n)l
Delete Ollog n)olog n)

Applications:

Red black tree offer worst case guarantee for insertion time, deletion time and search time. Not
only
does this make them valuable in time sensitive applications such as real time applications but it makes
them valuable building blocks in other data structures which provide worst case guarantees.

1. Most of the self-balancing BST library funetions like map and set in C++ (OR TreeSet and
TreeMap in Java) use Red Black Tree
2 It is used to implement CPU Scheduling Linux. Completely Fair Scheduler uses it.
 UNIT 4

UNIT IV ALGORITHM DESIGN TECHNIQUES

Dynamic Programming: Matrix-Chain Multiplication – Elements of Dynamic Programming – Longest
Common Subsequence- Greedy Algorithms: – Elements of the Greedy Strategy- An Activity-Selection
Problem - Huffman Coding.

Dynamic Programming:

Dynamic programming is a technique that breaks the problems into sub-problems, and saves the result for future
purposes so that we do not need to compute the result again. The sub problems are optimized to optimize the overall
solution is known as optimal substructure property. The main use of dynamic programming is to solve optimization
problems. Here, optimization problems mean that when we are trying to find out the minimum or the maximum solution
of a problem. The dynamic programming guarantees to find the optimal solution of a problem if the solution exists.

The definition of dynamic programming says that it is a technique for solving a complex problem by first breaking into
a collection of simpler subproblems, solving each subproblem just once, and then storing their solutions to avoid
repetitive computations.

Let's understand this approach through an example.

Consider an example of the Fibonacci series. The following series is the Fibonacci series:

0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, ,…

The numbers in the above series are not randomly calculated. Mathematically, we could write each of the terms using
the below formula:

F(n) = F(n-1) + F(n-2),

With the base values F(0) = 0, and F(1) = 1. To calculate the other numbers, we follow the above relationship. For
example, F(2) is the sum f(0) and f(1), which is equal to 1.

How can we calculate F(20)?

The F(20) term will be calculated using the nth formula of the Fibonacci series. The below figure shows that how F(20)
is calculated.

As we can observe in the above figure that F(20) is calculated as the sum of F(19) and F(18). In the dynamic
programming approach, we try to divide the problem into the similar subproblems. We are following this approach in
the above case where F(20) into the similar subproblems, i.e., F(19) and F(18). If we recap the definition of dynamic
Page 1
 UNIT 4

programming that it says the similar subproblem should not be computed more than once. Still, in the above case, the
subproblem is calculated twice. In the above example, F(18) is calculated two times; similarly, F(17) is also calculated
twice. However, this technique is quite useful as it solves the similar subproblems, but we need to be cautious while
storing the results because we are not particular about storing the result that we have computed once, then it can lead to
a wastage of resources.

In the above example, if we calculate the F(18) in the right subtree, then it leads to the tremendous usage of resources
and decreases the overall performance.

The solution to the above problem is to save the computed results in an array. First, we calculate F(16) and F(17) and
save their values in an array. The F(18) is calculated by summing the values of F(17) and F(16), which are already saved
in an array. The computed value of F(18) is saved in an array. The value of F(19) is calculated using the sum of F(18),
and F(17), and their values are already saved in an array. The computed value of F(19) is stored in an array. The value
of F(20) can be calculated by adding the values of F(19) and F(18), and the values of both F(19) and F(18) are stored in
an array. The final computed value of F(20) is stored in an array.

How does the dynamic programming approach work?

The following are the steps that the dynamic programming follows:

o It breaks down the complex problem into simpler subproblems.

o It finds the optimal solution to these sub-problems.
o It stores the results of subproblems (memoization). The process of storing the results of subproblems is known
as memorization.
o It reuses them so that same sub-problem is calculated more than once.
o Finally, calculate the result of the complex problem.

The above five steps are the basic steps for dynamic programming. The dynamic programming is applicable that are
having properties such as:

Those problems that are having overlapping subproblems and optimal substructures. Here, optimal substructure means
that the solution of optimization problems can be obtained by simply combining the optimal solution of all the
subproblems.

In the case of dynamic programming, the space complexity would be increased as we are storing the intermediate results,
but the time complexity would be decreased.

Approaches of dynamic programming

There are two approaches to dynamic programming:

o Top-down approach
o Bottom-up approach

Top-down approach

The top-down approach follows the memorization technique, while bottom-up approach follows the tabulation method.
Here memorization is equal to the sum of recursion and caching. Recursion means calling the function itself, while
caching means storing the intermediate results.

Advantages
Page 2
 UNIT 4

o It is very easy to understand and implement.

o It solves the subproblems only when it is required.
o It is easy to debug.

Disadvantages

It uses the recursion technique that occupies more memory in the call stack. Sometimes when the recursion is too deep,
the stack overflow condition will occur.

It occupies more memory that degrades the overall performance.

Let's understand dynamic programming through an example.

int fib(int n)
{
if(n<0)
error;
if(n==0)
return 0;
if(n==1)
return 1;
sum = fib(n-1) + fib(n-2);
}

In the above code, we have used the recursive approach to find out the Fibonacci series. When the value of 'n' increases,
the function calls will also increase, and computations will also increase. In this case, the time complexity increases
exponentially, and it becomes 2n.

One solution to this problem is to use the dynamic programming approach. Rather than generating the recursive tree
again and again, we can reuse the previously calculated value. If we use the dynamic programming approach, then the
time complexity would be O(n).

When we apply the dynamic programming approach in the implementation of the Fibonacci series, then the code would
look like:

static int count = 0;

int fib(int n)
{
if(memo[n]!= NULL)
return memo[n];
count++;
if(n<0)
error;
if(n==0)
return 0;
if(n==1)
return 1;
Page 3
 UNIT 4

sum = fib(n-1) + fib(n-2);

memo[n] = sum;
}

In the above code, we have used the memorization technique in which we store the results in an array to reuse the values.
This is also known as a top-down approach in which we move from the top and break the problem into sub-problems.

Bottom-Up approach

The bottom-up approach is also one of the techniques which can be used to implement the dynamic programming. It
uses the tabulation technique to implement the dynamic programming approach. It solves the same kind of problems
but it removes the recursion. If we remove the recursion, there is no stack overflow issue and no overhead of the recursive
functions. In this tabulation technique, we solve the problems and store the results in a matrix.

There are two ways of applying dynamic programming:

o Top-Down
o Bottom-Up

The bottom-up is the approach used to avoid the recursion, thus saving the memory space. The bottom-up is an algorithm
that starts from the beginning, whereas the recursive algorithm starts from the end and works backward. In the bottom-
up approach, we start from the base case to find the answer for the end. As we know, the base cases in the Fibonacci
series are 0 and 1. Since the bottom approach starts from the base cases, so we will start from 0 and 1.

Key points

o We solve all the smaller sub-problems that will be needed to solve the larger sub-problems then move to the
larger problems using smaller sub-problems.
o We use for loop to iterate over the sub-problems.
o The bottom-up approach is also known as the tabulation or table filling method.

Let's understand through an example.

Suppose we have an array that has 0 and 1 values at a[0] and a[1] positions, respectively shown as below:

Since the bottom-up approach starts from the lower values, so the values at a[0] and a[1] are added to find the value of
a[2] shown as below:

The value of a[3] will be calculated by adding a[1] and a[2], and it becomes 2 shown as below:

Page 4
 UNIT 4

The value of a[4] will be calculated by adding a[2] and a[3], and it becomes 3 shown as below:

The value of a[5] will be calculated by adding the values of a[4] and a[3], and it becomes 5 shown as below:

The code for implementing the Fibonacci series using the bottom-up approach is given below:

int fib(int n)
{
int A[];
A[0] = 0, A[1] = 1;
for( i=2; i<=n; i++)
{
A[i] = A[i-1] + A[i-2]
}
return A[n];
}

In the above code, base cases are 0 and 1 and then we have used for loop to find other values of Fibonacci series.

Let's understand through the diagrammatic representation.

Initially, the first two values, i.e., 0 and 1 can be represented as:

When i=2 then the values 0 and 1 are added shown as below:

Page 5
 UNIT 4

When i=3 then the values 1and 1 are added shown as below:

When i=4 then the values 2 and 1 are added shown as below:

When i=5, then the values 3 and 2 are added shown as below:

In the above case, we are starting from the bottom and reaching to the top.
Matrix Chain Multiplication:

Matrix chain multiplication (or Matrix Chain Ordering Problem, MCOP) is an optimization problem that to find the
most efficient way to multiply a given sequence of matrices. The problem is not actually to perform the
multiplications but merely to decide the sequence of the matrix multiplications involved.

It is a Method under Dynamic Programming in which previous output is taken as input for next.
Page 6
 UNIT 4

Here, Chain means one matrix's column is equal to the second matrix's row [always].

In general:

If A = ⌊aij⌋ is a p x q matrix
B = ⌊bij⌋ is a q x r matrix
C = ⌊cij⌋ is a p x r matrix

Then

Given following matrices {A1,A2,A3,...An} and we have to perform the matrix multiplication, which can be
accomplished by a series of matrix multiplications

A1 xA2 x,A3 x.....x An

Matrix Multiplication operation is associative in nature rather commutative. By this, we mean that we have to follow
the above matrix order for multiplication but we are free to parenthesize the above multiplication depending upon our
need.

In general, for 1≤ i≤ p and 1≤ j ≤ r

It can be observed that the total entries in matrix 'C' is 'pr' as the matrix is of dimension p x r Also each entry takes O
(q) times to compute, thus the total time to compute all possible entries for the matrix 'C' which is a multiplication of 'A'
and 'B' is proportional to the product of the dimension p q r.

It is also noticed that we can save the number of operations by reordering the parenthesis.

Example1: Let us have 3 matrices, A1,A2,A3 of order (10 x 100), (100 x 5) and (5 x 50) respectively.

Three Matrices can be multiplied in two ways:

1. A1,(A2,A3): First multiplying(A2 and A3) then multiplying and resultant withA1.
2. (A1,A2),A3: First multiplying(A1 and A2) then multiplying and resultant withA3.

No of Scalar multiplication in Case 1 will be:

• (100 x 5 x 50) + (10 x 100 x 50) = 25000 + 50000 = 75000

No of Scalar multiplication in Case 2 will be:

• (100 x 10 x 5) + (10 x 5 x 50) = 5000 + 2500 = 7500

To find the best possible way to calculate the product, we could simply parenthesis the expression in every possible
fashion and count each time how many scalar multiplication are required.

Matrix Chain Multiplication Problem can be stated as "find the optimal parenthesization of a chain of matrices to be
multiplied such that the number of scalar multiplication is minimized".

Page 7
 UNIT 4

Number of ways for parenthesizing the matrices:

There are very large numbers of ways of parenthesizing these matrices. If there are n items, there are (n-1) ways in
which the outer most pair of parenthesis can place.

(A1) (A2,A3,A4,................An)
Or (A1,A2) (A3,A4 .................An)
Or (A1,A2,A3) (A4 ...............An)
........................

Or(A1,A2,A3.............An-1) (An)

It can be observed that after splitting the kth matrices, we are left with two parenthesized sequence of matrices: one
consist 'k' matrices and another consist 'n-k' matrices.

Now there are 'L' ways of parenthesizing the left sublist and 'R' ways of parenthesizing the right sublist then the Total
will be L.R:

Also p (n) = c (n-1) where c (n) is the nth Catalon number

c (n) =

On applying Stirling's formula we have

c (n) = Ω

Which shows that 4n grows faster, as it is an exponential function, then n1.5.

Development of Dynamic Programming Algorithm

1. Characterize the structure of an optimal solution.

2. Define the value of an optimal solution recursively.
3. Compute the value of an optimal solution in a bottom-up fashion.
4. Construct the optimal solution from the computed information.

Dynamic Programming Approach

Let Ai,j be the result of multiplying matrices i through j. It can be seen that the dimension of Ai,j is pi-1 x pj matrix.

Dynamic Programming solution involves breaking up the problems into subproblems whose solution can be combined
to solve the global problem.

Page 8
 UNIT 4

At the greatest level of parenthesization, we multiply two matrices

A1.....n=A1....k x Ak+1....n)

Thus we are left with two questions:

o How to split the sequence of matrices?

o How to parenthesize the subsequence A1.....k andAk+1......n?

One possible answer to the first question for finding the best value of 'k' is to check all possible choices of 'k' and
consider the best among them. But that it can be observed that checking all possibilities will lead to an exponential
number of total possibilities. It can also be noticed that there exists only O (n2 ) different sequence of matrices, in this
way do not reach the exponential growth.

Step1: Structure of an optimal parenthesization: Our first step in the dynamic paradigm is to find the optimal
substructure and then use it to construct an optimal solution to the problem from an optimal solution to subproblems.

Let Ai....j where i≤ j denotes the matrix that results from evaluating the product

Ai Ai+1....Aj.

If i < j then any parenthesization of the product Ai Ai+1 ......Aj must split that the product between Ak and Ak+1 for some
integer k in the range i ≤ k ≤ j. That is for some value of k, we first compute the matrices Ai.....k & Ak+1....j and then
multiply them together to produce the final product Ai....j. The cost of computing Ai....k plus the cost of computing
Ak+1....j plus the cost of multiplying them together is the cost of parenthesization.

Step 2: A Recursive Solution: Let m [i, j] be the minimum number of scalar multiplication needed to compute the
matrixAi....j.

If i=j the chain consist of just one matrix Ai....i=Ai so no scalar multiplication are necessary to compute the product. Thus
m [i, j] = 0 for i= 1, 2, 3....n.

If i<j we assume that to optimally parenthesize the product we split it between Ak and Ak+1 where i≤ k ≤j. Then m [i,j]
equals the minimum cost for computing the subproducts Ai....k and Ak+1....j+ cost of multiplying them together. We know
Ai has dimension pi-1 x pi, so computing the product Ai....k and Ak+1....jtakes pi-1 pk pj scalar multiplication, we obtain

m [i,j] = m [i, k] + m [k + 1, j] + pi-1 pk pj

There are only (j-1) possible values for 'k' namely k = i, i+1.....j-1. Since the optimal parenthesization must use one of
these values for 'k' we need only check them all to find the best.

So the minimum cost of parenthesizing the product Ai Ai+1......Aj becomes

To construct an optimal solution, let us define s [i,j] to be the value of 'k' at which we can split the product
Ai Ai+1 .....Aj To obtain an optimal parenthesization i.e. s [i, j] = k such that

m [i,j] = m [i, k] + m [k + 1, j] + pi-1 pk pj

Page 9
 UNIT 4

Longest Common Subsequence Algorithm

A subsequence of a given sequence is just the given sequence with some elements left out.

Given two sequences X and Y, we say that the sequence Z is a common sequence of X and Y if Z is a subsequence of
both X and Y.

In the longest common subsequence problem, we are given two sequences X = (x1 x2....xm) and Y = (y1 y2 yn) and wish
to find a maximum length common subsequence of X and Y. LCS Problem can be solved using dynamic programming.

Characteristics of Longest Common Sequence

A brute-force approach we find all the subsequences of X and check each subsequence to see if it is also a subsequence
of Y, this approach requires exponential time making it impractical for the long sequence.

The longest common subsequence (LCS) is defined as the longest subsequence that is common to all the given

sequences, provided that the elements of the subsequence are not required to occupy consecutive positions within the

original sequences.

If S1 and S2 are the two given sequences then, Z is the common subsequence of S1 and S2 if Z is a subsequence of

both S1 and S2. Furthermore, Z must be a strictly increasing sequence of the indices of both S1 and S2.

In a strictly increasing sequence, the indices of the elements chosen from the original sequences must be in ascending
order in Z.

Given a sequence X = (x1 x2.....xm) we define the ith prefix of X for i=0, 1, and 2...m as Xi= (x1 x2.....xi). For example:
if X = (A, B, C, B, C, A, B, C) then X4= (A, B, C, B)

Optimal Substructure of an LCS: Let X = (x1 x2....xm) and Y = (y1 y2.....) yn) be the sequences and let Z = (z1 z2......zk)
be any LCS of X and Y.

o If xm = yn, then zk=x_m=yn and Zk-1 is an LCS of Xm-1and Yn-1

o If xm ≠ yn, then zk≠ xm implies that Z is an LCS of Xm-1and Y.
o If xm ≠ yn, then zk≠yn implies that Z is an LCS of X and Yn-1

Step 2: Recursive Solution: LCS has overlapping subproblems property because to find LCS of X and Y, we may need
to find the LCS of Xm-1 and Yn-1. If xm ≠ yn, then we must solve two subproblems finding an LCS of X and Yn-1.Whenever
of these LCS's longer is an LCS of x and y. But each of these subproblems has the subproblems of finding the LCS of
Xm-1 and Yn-1.

Let c [i,j] be the length of LCS of the sequence Xiand Yj.If either i=0 and j =0, one of the sequences has length 0, so the
LCS has length 0. The optimal substructure of the LCS problem given the recurrence formula

Page 10
 UNIT 4

Step3: Computing the length of an LCS: let two sequences X = (x1 x2.....xm) and Y = (y1 y2..... yn) as inputs. It stores
the c [i,j] values in the table c [0......m,0..........n].Table b [1..........m, 1..........n] is maintained which help us to construct
an optimal solution. c [m, n] contains the length of an LCS of X,Y.

Greedy Algorithm Tutorial – Examples, Application and

Practice Problem
Greedy Algorithm is defined as a method for solving optimization problems by taking
decisions that result in the most evident and immediate benefit irrespective of the final outcome.
It works for cases where minimization or maximization leads to the required solution.

Table of Content
• What is Greedy Algorithm?
• Characteristics of Greedy Algorithm
• Examples of Greedy Algorithm
• Why to use Greedy Approach?
• How does the Greedy Algorithm works?
• Greedy Algorithm Vs Dynamic Programming
• Applications of Greedy Algorithms
• Advantages of Greedy Algorithms
• Disadvantages of the Greedy Approach
• Greedy Algorithm Most Asked Interview Problems
• Frequently Asked Questions on Greedy Algorithm
What is Greedy Algorithm?
A greedy algorithm is a problem-solving technique that makes the best local choice at each
step in the hope of finding the global optimum solution. It prioritizes immediate benefits over
long-term consequences, making decisions based on the current situation without considering
future implications. While this approach can be efficient and straightforward, it doesn’t
guarantee the best overall outcome for all problems.
However, it’s important to note that not all problems are suitable for greedy algorithms. They
work best when the problem exhibits the following properties:
• Greedy Choice Property: The optimal solution can be constructed by making the
best local choice at each step.
• Optimal Substructure: The optimal solution to the problem contains the optimal
solutions to its subproblems.
Characteristics of Greedy Algorithm
Here are the characteristics of a greedy algorithm:
• Greedy algorithms are simple and easy to implement.
• They are efficient in terms of time complexity, often providing quick solutions.
• Greedy algorithms are used for optimization problems where a locally optimal choice
leads to a globally optimal solution.
• These algorithms do not reconsider previous choices, as they make decisions based on
current information without looking ahead.
• Greedy algorithms are suitable for problems for optimal substructure.
These characteristics help to define the nature and usage of greedy algorithms in problem-
solving.
Examples of Greedy Algorithm
Page 11
 UNIT 4

Several well-known algorithms fall under the category of greedy algorithms. Here are a few
examples:
• Dijkstra’s Algorithm: This algorithm finds the shortest path between two nodes in a
graph. It works by repeatedly choosing the shortest edge available from the current
node.
• Kruskal’s Algorithm: This algorithm finds the minimum spanning tree of a graph. It
works by repeatedly choosing the edge with the minimum weight that does not create
a cycle.
• Fractional Knapsack Problem: This problem involves selecting items with the
highest value-to-weight ratio to fill a knapsack with a limited capacity. The greedy
algorithm selects items in decreasing order of their value-to-weight ratio until the
knapsack is full.
• Scheduling and Resource Allocation : The greedy algorithm can be used to schedule
jobs or allocate resources in an efficient manner.
• Coin Change Problem : The greedy algorithm can be used to make change for a
given amount with the minimum number of coins, by always choosing the coin with
the highest value that is less than the remaining amount to be changed.
• Huffman Coding : The greedy algorithm can be used to generate a prefix-free code
for data compression, by constructing a binary tree in a way that the frequency of each
character is taken into consideration.
Want to master Greedy algorithm and more? Check out our DSA Self-Paced Course for a
comprehensive guide to Data Structures and Algorithms at your own pace. This course will
help you build a strong foundation and advance your problem-solving skills.
Why to use Greedy Approach?
Here are some reasons why you might use the Greedy Approach:
• Simple and easy to understand: The Greedy Approach is straightforward and easy to
implement, making it a good choice for beginners.
• Fast and efficient: It usually finds a solution quickly, making it suitable for problems
where time is a constraint.
• Provides a good enough solution: While not always optimal, the Greedy Approach
often finds a solution that is close to the best possible solution.
• Can be used as a building block for other algorithms: The Greedy Approach can
be used as a starting point for developing more complex algorithms.
• Useful for a variety of problems: The Greedy Approach can be applied to a wide
range of optimization problems, including knapsack problems, scheduling problems,
and routing problems.
However, it’s important to remember that the Greedy Approach doesn’t always find the optimal
solution . There are cases where it can lead to suboptimal solutions. Therefore, it is necessary to
carefully consider the problem and the potential drawbacks before using the Greedy Approach.
How does the Greedy Algorithm works?
Greedy Algorithm solve optimization problems by making the best local choice at each step in
the hope of finding the global optimum. It’s like taking the best option available at each
moment, hoping it will lead to the best overall outcome.
Here’s how it works:
1. Start with the initial state of the problem. This is the starting point from where you
begin making choices.
2. Evaluate all possible choices you can make from the current state. Consider all the
options available at that specific moment.

Page 12
 UNIT 4

3. Choose the option that seems best at that moment, regardless of future consequences.
This is the “greedy” part – you take the best option available now, even if it might not
be the best in the long run.
4. Move to the new state based on your chosen option. This becomes your new starting
point for the next iteration.
5. Repeat steps 2-4 until you reach the goal state or no further progress is possible. Keep
making the best local choices until you reach the end of the problem or get stuck..
Example:
Let’s say you have a set of coins with values {1, 2, 5, 10, 20, 50, 100} and you need to give
minimum number of coin to someone change for 36 .
The greedy algorithm for making change would work as follows:
1. Start with the largest coin value that is less than or equal to the amount to be
changed. In this case, the largest coin less than 36 is 20 .
2. Subtract the largest coin value from the amount to be changed, and add the coin to the
solution. In this case, subtracting 20 from 36 gives 16 , and we add a 20 coin to the
solution.
3. Repeat steps 1 and 2 until the amount to be changed becomes 0.
So, using the greedy algorithm, the solution for making change for 36 would be one 20 coins,
one 10 coin, one 5 coins and one 1 coin needed.
Note: This is just one example, and other greedy choices could have been made at each step.
However, in this case, the greedy approach leads to the optimal solution.
The greedy algorithm is not always the optimal solution for every optimization problem, as
shown in the example below.
• One such example where the Greedy Approach fails is to find the Maximum weighted
path of nodes in the given graph .

Graph with weighted vertices

• In the above graph starting from the root node 10 if we greedily select the next node to
obtain the most weighted path the next selected node will be 5 that will take the total
sum to 15 and the path will end as there is no child of 5 but the path 10 -> 5 is not the
maximum weight path.

Page 13
 UNIT 4

Greedy Approach fails

• In order to find the most weighted path all possible path sum must be computed and
their path sum must be compared to get the desired result, it is visible that the most
weighted path in the above graph is 10 -> 1 -> 30 that gives the path sum 41 .

Correct Approach

• In such cases Greedy approach wouldn’t work instead complete paths

from root to leaf node has to be considered to get the correct answer i.e. the most
weighted path, This can be achieved by recursively checking all the paths and
calculating their weight.
Greedy Algorithm Vs Dynamic Programming
Below are the comparison of Greedy Algorithm and Dynamic Programming based on various
criteria:
Criteria Greedy Algorithm Dynamic Programming

Makes the locally optimal choice at Solves subproblems and builds up to

Basic Idea each stage the optimal solution

Page 14
 UNIT 4

Criteria Greedy Algorithm Dynamic Programming

Not always guaranteed to provide Guarantees the globally optimal

Optimal Solution the globally optimal solution solution

Typically, faster; often linear or Usually, slower due to solving

Time Complexity polynomial time overlapping subproblems

Requires less memory; often Requires more memory due to storing

Space Complexity constant or linear space intermediate results

Subproblems Does not handle overlapping Handles overlapping subproblems

Overlapping subproblems efficiently

Finding minimum spanning tree, Matrix chain multiplication, shortest

Examples Huffman coding path problems

Used when a greedy choice at each Applied when the problem can be
step leads to the globally optimal broken down into overlapping
Applications solution subproblems

Applications of Greedy Algorithms:

• Dijkstra’s shortest path algorithm: Finds the shortest path between two nodes in a
graph.
• Kruskal’s minimum spanning tree algorithm: Finds the minimum spanning tree for
a weighted graph.
• Huffman coding: Creates an optimal prefix code for a set of symbols based on their
frequencies.
• Fractional knapsack problem: Determines the most valuable items to carry in a
knapsack with a limited weight capacity.
• Activity selection problem: Chooses the maximum number of non-overlapping
activities from a set of activities.
Advantages of Greedy Algorithms:
• Simple and easy to understand: Greedy algorithms are often straightforward to
implement and reason about.
• Efficient for certain problems: They can provide optimal solutions for specific
problems, like finding the shortest path in a graph with non-negative edge weights.
• Fast execution time: Greedy algorithms generally have lower time complexity
compared to other algorithms for certain problems.

Page 15
 UNIT 4

• Intuitive and easy to explain: The decision-making process in a greedy algorithm is

often easy to understand and justify.
• Can be used as building blocks for more complex algorithms: Greedy algorithms
can be combined with other techniques to design more sophisticated algorithms for
challenging problems.
Disadvantages of the Greedy Approach:
• Not always optimal: Greedy algorithms prioritize local optima over global optima,
leading to suboptimal solutions in some cases.
• Difficult to prove optimality: Proving the optimality of a greedy algorithm can be
challenging, requiring careful analysis.
• Sensitive to input order: The order of input data can affect the solution generated by
a greedy algorithm.
• Limited applicability: Greedy algorithms are not suitable for all problems and may
not be applicable to problems with complex constraints.

Huffman Coding | Greedy Algo-3

Huffman coding is a lossless data compression algorithm. The idea is to assign variable-length
codes to input characters, lengths of the assigned codes are based on the frequencies of
corresponding characters.
The variable-length codes assigned to input characters are Prefix Codes, means the codes (bit
sequences) are assigned in such a way that the code assigned to one character is not the prefix of
code assigned to any other character. This is how Huffman Coding makes sure that there is no
ambiguity when decoding the generated bitstream.
Let us understand prefix codes with a counter example. Let there be four characters a, b, c and d,
and their corresponding variable length codes be 00, 01, 0 and 1. This coding leads to ambiguity
because code assigned to c is the prefix of codes assigned to a and b. If the compressed bit
stream is 0001, the de-compressed output may be “cccd” or “ccb” or “acd” or “ab”.
See this for applications of Huffman Coding.
There are mainly two major parts in Huffman Coding
1. Build a Huffman Tree from input characters.
2. Traverse the Huffman Tree and assign codes to characters.

Algorithm:

The method which is used to construct optimal prefix code is called Huffman coding.
This algorithm builds a tree in bottom up manner. We can denote this tree by T
Let, |c| be number of leaves
|c| -1 are number of operations required to merge the nodes. Q be the priority queue which can
be used while constructing binary heap.
Algorithm Huffman (c)
{
n= |c|
Q = c
for i<-1 to n-1
Page 16
 UNIT 4

do
{
temp <- get node ()
left (temp] Get_min (Q) right [temp] Get Min (Q)
a = left [templ b = right [temp]
F [temp]<- f[a] + [b]
insert (Q, temp)
}
return Get_min (0)
}
Steps to build Huffman Tree
Input is an array of unique characters along with their frequency of occurrences and output is
Huffman Tree.
1. Create a leaf node for each unique character and build a min heap of all leaf nodes
(Min Heap is used as a priority queue. The value of frequency field is used to
compare two nodes in min heap. Initially, the least frequent character is at root)
2. Extract two nodes with the minimum frequency from the min heap.

3. Create a new internal node with a frequency equal to the sum of the two nodes
frequencies. Make the first extracted node as its left child and the other extracted
node as its right child. Add this node to the min heap.
4. Repeat steps#2 and #3 until the heap contains only one node. The remaining node is
the root node and the tree is complete.
Let us understand the algorithm with an example:
character Frequency
a 5
b 9
c 12
d 13
e 16
f 45
Step 1. Build a min heap that contains 6 nodes where each node represents root of a tree with
single node.
Step 2 Extract two minimum frequency nodes from min heap. Add a new internal node with
frequency 5 + 9 = 14.

Illustration of step 2
Page 17
 UNIT 4

Now min heap contains 5 nodes where 4 nodes are roots of trees with single element each, and
one heap node is root of tree with 3 elements
character Frequency
c 12
d 13
Internal Node 14
e 16
f 45
Step 3: Extract two minimum frequency nodes from heap. Add a new internal node with
frequency 12 + 13 = 25

Illustration of step 3

Now min heap contains 4 nodes where 2 nodes are roots of trees with single element each, and
two heap nodes are root of tree with more than one nodes
character Frequency
Internal Node 14
e 16
Internal Node 25
f 45
Step 4: Extract two minimum frequency nodes. Add a new internal node with frequency 14 +
16 = 30

Illustration of step 4

Now min heap contains 3 nodes.

Page 18
 UNIT 4

character Frequency
Internal Node 25
Internal Node 30
f 45
Step 5: Extract two minimum frequency nodes. Add a new internal node with frequency 25 +
30 = 55

Illustration of step 5

Now min heap contains 2 nodes.

character Frequency
f 45
Internal Node 55
Step 6: Extract two minimum frequency nodes. Add a new internal node with frequency 45 +
55 = 100

Illustration of step 6

Now min heap contains only one node.

character Frequency
Page 19
 UNIT 4

Internal Node 100

Since the heap contains only one node, the algorithm stops here.
Steps to print codes from Huffman Tree:
Traverse the tree formed starting from the root. Maintain an auxiliary array. While moving to
the left child, write 0 to the array. While moving to the right child, write 1 to the array. Print
the array when a leaf node is encountered.

Steps to print code from HuffmanTree

The codes are as follows:

character code-word
f 0
c 100
d 101
a 1100
b 1101
e 111
Below is the implementation of above approach:

import java.util.Comparator;
import java.util.PriorityQueue;
import java.util.Scanner;

class Huffman {

// recursive function to print the

Page 20
 UNIT 4

// huffman-code through the tree traversal.

// Here s is the huffman - code generated.
public static void printCode(HuffmanNode root, String s)
{

// base case; if the left and right are null

// then its a leaf node and we print
// the code s generated by traversing the tree.
if (root.left == null && root.right == null
&& Character.isLetter(root.c)) {

// c is the character in the node

System.out.println(root.c + ":" + s);

return;
}

// if we go to left then add "0" to the code.

// if we go to the right add"1" to the code.

// recursive calls for left and

// right sub-tree of the generated tree.
printCode(root.left, s + "0");
printCode(root.right, s + "1");
}

// main function
public static void main(String[] args)
{
Scanner s = new Scanner(System.in);
// number of characters.
int n = 6;
char[] charArray = { 'a', 'b', 'c', 'd', 'e', 'f' };
int[] charfreq = { 5, 9, 12, 13, 16, 45 };
// creating a priority queue q.
// makes a min-priority queue(min-heap).
PriorityQueue<HuffmanNode> q
= new PriorityQueue<HuffmanNode>(
n, new MyComparator());
for (int i = 0; i < n; i++) {
// creating a Huffman node object
// and add it to the priority queue.
HuffmanNode hn = new HuffmanNode();
hn.c = charArray[i];
hn.data = charfreq[i];
hn.left = null;
hn.right = null;
// add functions adds
// the huffman node to the queue.
q.add(hn);
}
// create a root node
HuffmanNode root = null;
// Here we will extract the two minimum value
// from the heap each time until
// its size reduces to 1, extract until
// all the nodes are extracted.
while (q.size() > 1) {
// first min extract.
HuffmanNode x = q.peek();
q.poll();
// second min extract.
HuffmanNode y = q.peek();
q.poll();

Page 21
 UNIT 4

// new node f which is equal

HuffmanNode f = new HuffmanNode();
// to the sum of the frequency of the two nodes
// assigning values to the f node.
f.data = x.data + y.data;
f.c = '-';
// first extracted node as left child.
f.left = x;

// second extracted node as the right child.

f.right = y;
// marking the f node as the root node.
root = f;
// add this node to the priority-queue.
q.add(f);
}
// print the codes by traversing the tree
printCode(root, "");
}
}
// node class is the basic structure
// of each node present in the Huffman - tree.
class HuffmanNode {
int data;
char c;
HuffmanNode left;
HuffmanNode right;
}
// comparator class helps to compare the node
// on the basis of one of its attribute.
// Here we will be compared
// on the basis of data values of the nodes.
class MyComparator implements Comparator<HuffmanNode> {
public int compare(HuffmanNode x, HuffmanNode y)
{
return x.data - y.data;
}
}
// This code is contributed by Kunwar Desh Deepak Singh

Output

f: 0
c: 100
d: 101
a: 1100
b: 1101
e: 111
Time complexity: O(nlogn) where n is the number of unique characters. If there are n nodes,
extractMin() is called 2*(n – 1) times. extractMin() takes O(logn) time as it calls
minHeapify(). So, the overall complexity is O(nlogn).
If the input array is sorted, there exists a linear time algorithm. We will soon be discussing this
in our next post.
Space complexity :- O(N)
Applications of Huffman Coding:
1. They are used for transmitting fax and text.
2. They are used by conventional compression formats like PKZIP, GZIP, etc.
Page 22
 UNIT 4

3. Multimedia codecs like JPEG, PNG, and MP3 use Huffman encoding(to be more
precise the prefix codes).
It is useful in cases where there is a series of frequently occurring characters.

2 mark

Memorization: What if we stored sub-problems and used the stored solutions in a recursive
algorithm? This is like divide-and-conquer, top down, but should benefit like DP which is
bottom-up. Memorized version maintains an entry in a table. One can use a fixed table or a
hash table.

Page 23
 UNIT 5
UNIT V
NP COMPLETE AND NP HARD
NP-Completeness: Polynomial Time – Polynomial-Time Verification – NP- Completeness
and Reducibility – NP-Completeness Proofs – NP-Complete Problems.

Introduction to NP-Complete Complexity Classes

NP-complete problems are a subset of the larger class of NP (nondeterministic polynomial

time) problems. NP problems are a class of computational problems that can be solved in
polynomial time by a non-deterministic machine and can be verified in polynomial time by a
deterministic Machine. A problem L in NP is NP-complete if all other problems in NP can be
reduced to L in polynomial time. If any NP-complete problem can be solved in polynomial
time, then every problem in NP can be solved in polynomial time. NP-complete problems are
the hardest problems in the NP set.
A decision problem L is NP-complete if it follow the below two properties:
1. L is in NP (Any solution to NP-complete problems can be checked quickly, but no
efficient solution is known).
2. Every problem in NP is reducible to L in polynomial time (Reduction is defined
below).
A problem is NP-Hard if it obeys Property 2 above and need not obey Property 1. Therefore, a
problem is NP-complete if it is both NP and NP-hard.

NP-Complete Complexity Classes

Decision vs Optimization Problems

NP-completeness applies to the realm of decision problems. It was set up this way because it’s
easier to compare the difficulty of decision problems than that of optimization problems. In
reality, though, being able to solve a decision problem in polynomial time will often permit us to
solve the corresponding optimization problem in polynomial time (using a polynomial number
of calls to the decision problem). So, discussing the difficulty of decision problems is often
really equivalent to discussing the difficulty of optimization problems.
For example, consider the vertex cover problem (Given a graph, find out the minimum sized
vertex set that covers all edges). It is an optimization problem. The corresponding decision
problem is, given undirected graph G and k, is there a vertex cover of size k?
What is Reduction?
Let L1 and L2 be two decision problems. Suppose algorithm A2 solves L2. That is, if y is an
input for L2 then algorithm A2 will answer Yes or No depending upon whether y belongs
to L2 or not.
The idea is to find a transformation from L1 to L2 so that algorithm A2 can be part of
Page 1
 UNIT 5
algorithm A1 to solve L1.

Learning reduction, in general, is very important. For example, if we have library functions to
solve certain problems and if we can reduce a new problem to one of the solved problems, we
save a lot of time. Consider the example of a problem where we have to find the minimum
product path in a given directed graph where the product of the path is the multiplication of
weights of edges along the path. If we have code for Dijkstra’s algorithm to find the shortest
path, we can take the log of all weights and use Dijkstra’s algorithm to find the minimum
product path rather than writing a fresh code for this new problem.
How to prove that a given problem is NP-complete?
From the definition of NP-complete, it appears impossible to prove that a problem L is NP-
Complete. By definition, it requires us to that show every problem in NP is polynomial time
reducible to L. Fortunately, there is an alternate way to prove it. The idea is to take a known
NP-Complete problem and reduce it to L. If a polynomial-time reduction is possible, we can
prove that L is NP-Complete by transitivity of reduction (If an NP-Complete problem is
reducible to L in polynomial time, then all problems are reducible to L in polynomial time).
What was the first problem proved as NP-Complete?
There must be some first NP-Complete problem proved by the definition of NP-Complete
problems. SAT (Boolean satisfiability problem) is the first NP-Complete problem proved by
Cook (See CLRS book for proof).
It is always useful to know about NP-Completeness even for engineers. Suppose you are asked
to write an efficient algorithm to solve an extremely important problem for your company. After
a lot of thinking, you can only come up exponential time approach which is impractical. If you
don’t know about NP-Completeness, you can only say that I could not come up with an efficient
algorithm. If you know about NP-Completeness and prove that the problem is NP-complete, you
can proudly say that the polynomial-time solution is unlikely to exist. If there is a polynomial-
time solution possible, then that solution solves a big problem of computer science many
scientists have been trying for years.
Difference between NP hard and NP complete
problem
Difference between NP-Hard and NP-Complete:
NP-hard NP-Complete

NP-Hard problems(say X) can be solved if NP-Complete problems can be solved by a non-

and only if there is a NP-Complete deterministic Algorithm/Turing Machine in
problem(say Y) that can be reducible into X polynomial time.
in polynomial time.

To solve this problem, it do not have to be in To solve this problem, it must be both NP and NP-
NP . hard problems.
Page 2
 UNIT 5

NP-hard NP-Complete

Do not have to be a Decision problem. It is exclusively a Decision problem.

Example: Determine whether a graph has a

Hamiltonian cycle, Determine whether a Boolean
Example: Halting problem, Vertex cover formula is satisfiable or not, Circuit-satisfiability
problem, etc. problem, etc.

Polynomial Time
An algorithm is said to be solvable in polynomial time if the number of steps required to
complete the algorithm for a given input is O (nk) for some nonnegative integer k, where n is
the complexity of the input. Polynomial-time algorithms are said to be "fast." Most familiar
mathematical operations such as addition, subtraction, multiplication, and division, as well as
computing square roots, powers, and logarithms, can be performed in polynomial time.
Computing the digits of most interesting mathematical constants, including 𝝅𝝅 and 𝒆𝒆 , can
also be done in polynomial time.

Polynomial Time Verification

Before talking about the class of NP-complete problems, it is essential to introduce the notion of a verification
algorithm.

Many problems are hard to solve, but they have the property that it easy to authenticate the solution if one is
provided.

Hamiltonian cycle problem:-

Consider the Hamiltonian cycle problem. Given an undirected graph G, does G have a cycle that visits each vertex
exactly once? There is no known polynomial time algorithm for this dispute.

Note: - It means you can't build a Hamiltonian cycle in a graph with a polynomial time even if there
is no specific path is given for the Hamiltonian cycle with the particular vertex, yet you can't verify
the Hamiltonian cycle within the polynomial time

Page 3
 UNIT 5
Fig: Hamiltonian Cycle

Let us understand that a graph did have a Hamiltonian cycle. It would be easy for someone to convince of this.
They would similarly say: "the period is hv3, v7, v1....v13i.

We could then inspect the graph and check that this is indeed a legal cycle and that it visits all of the vertices of
the graph exactly once. Thus, even though we know of no efficient way to solve the Hamiltonian cycle problem,
there is a beneficial way to verify that a given cycle is indeed a Hamiltonian cycle.

Note:-For the verification in the Polynomial-time of an undirected Hamiltonian cycle graph G. There
must be exact/specific/definite path must be given of Hamiltonian cycle then you can verify in the
polynomial time.

Definition of Certificate: - A piece of information which contains in the given path of a vertex is known as
certificate

Relation of P and NP classes

1. P contains in NP
2. P=NP

1. Observe that P contains in NP. In other words, if we can solve a problem in polynomial time, we can indeed verify
the solution in polynomial time. More formally, we do not need to see a certificate (there is no need to specify the
vertex/intermediate of the specific path) to solve the problem; we can explain it in polynomial time anyway.
2. However, it is not known whether P = NP. It seems you can verify and produce an output of the set of decision-
based problems in NP classes in a polynomial time which is impossible because according to the definition of NP
classes you can verify the solution within the polynomial time. So this relation can never be held.

Reductions:
The class NP-complete (NPC) problems consist of a set of decision problems (a subset of class NP) that no one
knows how to solve efficiently. But if there were a polynomial solution for even a single NP-complete problem,
then every problem in NPC will be solvable in polynomial time. For this, we need the concept of reductions.

Suppose there are two problems, A and B. You know that it is impossible to solve problem A in polynomial time.
You want to prove that B cannot be explained in polynomial time. We want to show that (A ∉ P) => (B ∉ P)

Consider an example to illustrate reduction: The following problem is well-known to be NPC:

3-color: Given a graph G, can each of its vertices be labeled with one of 3 different colors such that two adjacent
vertices do not have the same label (color).

Coloring arises in various partitioning issues where there is a constraint that two objects cannot be assigned to
the same set of partitions. The phrase "coloring" comes from the original application which was in map drawing.
Two countries that contribute a common border should be colored with different colors.

It is well known that planar graphs can be colored (maps) with four colors. There exists a polynomial time
algorithm for this. But deciding whether this can be done with 3 colors is hard, and there is no polynomial time
algorithm for it.

Page 4
 UNIT 5

Fig: Example of 3-colorable and non-3-colorable graphs.

Polynomial Time Reduction:

We say that Decision Problem L1 is Polynomial time Reducible to decision Problem L2 (L1≤p L2) if there is a
polynomial time computation function f such that of all x, xϵL1 if and only if xϵL2.

Introduction to NP-Complete Complexity Classes

NP-complete problems are a subset of the larger class of NP (nondeterministic polynomial
time) problems. NP problems are a class of computational problems that can be solved in
polynomial time by a non-deterministic machine and can be verified in polynomial time by a
deterministic Machine. A problem L in NP is NP-complete if all other problems in NP can be
reduced to L in polynomial time. If any NP-complete problem can be solved in polynomial
time, then every problem in NP can be solved in polynomial time. NP-complete problems are
the hardest problems in the NP set.
A decision problem L is NP-complete if it follow the below two properties:
1. L is in NP (Any solution to NP-complete problems can be checked quickly, but no
efficient solution is known).
2. Every problem in NP is reducible to L in polynomial time (Reduction is defined
below).
A problem is NP-Hard if it obeys Property 2 above and need not obey Property 1. Therefore, a
problem is NP-complete if it is both NP and NP-hard.

NP-Complete Complexity Classes

Decision vs Optimization Problems

Page 5
 UNIT 5
NP-completeness applies to the realm of decision problems. It was set up this way because it’s
easier to compare the difficulty of decision problems than that of optimization problems. In
reality, though, being able to solve a decision problem in polynomial time will often permit us to
solve the corresponding optimization problem in polynomial time (using a polynomial number
of calls to the decision problem). So, discussing the difficulty of decision problems is often
really equivalent to discussing the difficulty of optimization problems.
For example, consider the vertex cover problem (Given a graph, find out the minimum sized
vertex set that covers all edges). It is an optimization problem. The corresponding decision
problem is, given undirected graph G and k, is there a vertex cover of size k?
What is Reduction?
Let L1 and L2 be two decision problems. Suppose algorithm A2 solves L2. That is, if y is an
input for L2 then algorithm A2 will answer Yes or No depending upon whether y belongs
to L2 or not.
The idea is to find a transformation from L1 to L2 so that algorithm A2 can be part of
algorithm A1 to solve L1.

Learning reduction, in general, is very important. For example, if we have library functions to
solve certain problems and if we can reduce a new problem to one of the solved problems, we
save a lot of time. Consider the example of a problem where we have to find the minimum
product path in a given directed graph where the product of the path is the multiplication of
weights of edges along the path. If we have code for Dijkstra’s algorithm to find the shortest
path, we can take the log of all weights and use Dijkstra’s algorithm to find the minimum
product path rather than writing a fresh code for this new problem.
How to prove that a given problem is NP-complete?
From the definition of NP-complete, it appears impossible to prove that a problem L is NP-
Complete. By definition, it requires us to that show every problem in NP is polynomial time
reducible to L. Fortunately, there is an alternate way to prove it. The idea is to take a known
NP-Complete problem and reduce it to L. If a polynomial-time reduction is possible, we can
prove that L is NP-Complete by transitivity of reduction (If an NP-Complete problem is
reducible to L in polynomial time, then all problems are reducible to L in polynomial time).
What was the first problem proved as NP-Complete?
There must be some first NP-Complete problem proved by the definition of NP-Complete
problems. SAT (Boolean satisfiability problem) is the first NP-Complete problem proved by
Cook (See CLRS book for proof).
It is always useful to know about NP-Completeness even for engineers. Suppose you are asked
to write an efficient algorithm to solve an extremely important problem for your company. After
a lot of thinking, you can only come up exponential time approach which is impractical. If you
don’t know about NP-Completeness, you can only say that I could not come up with an efficient
algorithm. If you know about NP-Completeness and prove that the problem is NP-complete, you
can proudly say that the polynomial-time solution is unlikely to exist. If there is a polynomial-
time solution possible, then that solution solves a big problem of computer science many
scientists have been trying for years.
Page 6
 UNIT 5

28.12. NP-Completeness Proofs

28.12.1. NP-Completeness Proofs

To start the process of being able to prove problems are NP-complete, we need to prove just one
problem HH is NP-complete. After that, to show that any problem XX is NP-hard, we just need to
reduce HH to XX. When doing NP-completeness proofs, it is very important not to get this reduction
backwards! If we reduce candidate problem XX to known hard problem HH, this means that we use HH as a
step to solving XX. All that means is that we have found a (known) hard way to solve XX. However, when we
reduce known hard problem HH to candidate problem XX, that means we are using XX as a step to solve HH.
And if we know that HH is hard, that means XX must also be hard (because if XX were not hard, then neither
would HH be hard).

So a crucial first step to getting this whole theory off the ground is finding one problem that is NP-hard. The
first proof that a problem is NP-hard (and because it is in NP, therefore NP-complete) was done by Stephen
Cook. For this feat, Cook won the first Turing award, which is the closest Computer Science equivalent to the
Nobel Prize. The “grand-daddy” NP-complete problem that Cook used is called SATISFIABILITY (or SAT for
short).

A Boolean expression is comprised of Boolean variables combined using the operators AND (⋅⋅), OR (++),
and NOT (to negate Boolean variable xx we write 𝑥𝑥 �). A literal is a Boolean variable or its negation.
A clause is one or more literals OR’ed together. Let EE be a Boolean expression over
variables x1,x2,...,xnx1,x2,...,xn. Then we define Conjunctive Normal Form (CNF) to be a Boolean expression
written as a series of clauses that are AND’ed together. For example,

is in CNF, and has three clauses. Now we can define the problem SAT.

Problem

SATISFIABILITY (SAT)

Input: A Boolean expression EE over variables x1,x2,...x1,x2,... in Conjunctive Normal Form.

Output: YES if there is an assignment to the variables that makes EE true, NO otherwise.

Cook proved that SAT is NP-hard. Explaining Cook’s proof is beyond the scope of this course. But we can
briefly summarize it as follows. Any decision problem FF can be recast as some language acceptance
problem L:

That is, if a decision problem FF yields YES on input II, then there is a language LL containing
string I′I′ where I′I′ is some suitable transformation of input II. Conversely, if F would give answer NO for
input II, then II ‘s transformed version I′I′ is not in the language L.

Turing machines are a simple model of computation for writing programs that are language acceptors. There
is a “universal” Turing machine that can take as input a description for a Turing machine, and an input string,
and return the execution of that machine on that string. This Turing machine in turn can be cast as a Boolean
expression such that the expression is satisfiable if and only if the Turing machine yields ACCEPT for that
string. Cook used Turing machines in his proof because they are simple enough that he could develop this
transformation of Turing machines to Boolean expressions, but rich enough to be able to compute any
function that a regular computer can compute. The significance of this transformation is that any decision
problem that is performable by the Turing machine is transformable to SAT. Thus, SAT is NP-hard. Page 7
 UNIT 5
To show that a decision problem XX is NP-complete, we prove that XX is in NP (normally easy, and normally
done by giving a suitable polynomial-time, non-deterministic algorithm) and then prove that XX is NP-hard.
To prove that XX is NP-hard, we choose a known NP-complete problem, say AA. We describe a polynomial-
time transformation that takes an arbitrary instance II of AA to an instance I′I′ of XX. We then describe a
polynomial-time transformation from SLN′SLN′ to SLNSLN such that SLNSLN is the solution for II.

The following modules show a number of known NP-complete problems, and also some proofs that they are
NP-complete. The various proofs will link the problems together as shown here:

Figure We will use this sequence of reductions for the NP Complete Proof

NP-Complete proofs
This module 35 focuses on proving NP-Complete problems. The module extends previous
module by providing proof outlines for NP-Complete problems. The module also discusses
proofs for some problems like TSP, Clique, vertex cover and sum of subsets. The objectives of
this module are

• To Understand proof of NP-C

• To understand proof of TSP problem
• To understand proof of Clique, Vertex cover problem and Sum of Subsets

Computational Complexity

There are two types of complexity theory. One is related to algorithm complexity analysis
called algorithmic complexity theory and another related to problems called computational
complexity theory [2,3]. Computational complexity theory is different. Computational
complexity aims to determine lower bound on the efficiency of all the algorithms for a given
problem and Computational complexity is measured independently of the implementation. In
simple words, computational complexity is about the problem and not the algorithm.

NP-Complete Problems

Page 8
 UNIT 5
NP-Complete (or NPC) problems are a set of problems that are well connected. A problem x
that is in NP, if any one finds an polynomial time algorithm even for one problem, it implies that
polynomial time algorithm for all NP-Complete problems. In other words: Problem x is in NP,
then every problem in NP is reducible to problem x. Let us present the overview of reductions
first.

Reductions

Reduction algorithm reduces a problem to another problem. There are two types of reduction.
One is called Turing reduction and another is called Karp reduction. They are explained in module
34. Reductions form the basis of NP-Complete proofs. The proof of NP-Complete problems starts
with reducing a well-known NP-Complete problem to a problem for which NP-Complete proof is
sought. This reduction should take place in polynomial time. Then the equivalence of the
problems is justified. Let us discuss some basic NP-Complete proofs now:

Traveling Salesman Problem

Let us consider giving NP-Complete proof for travelling salesman (TSP) problem. One can
consider NP-Complete proof outline.

1. Take a well-known NP-Complete problem and reduce that to the given problem for which
proof is sought and prove that the reduction is done in polynomial time.
3. Argue that the given problem is as hard as the well known NP-Compete problem.

To prove TSP as NP-Complete, take a well-known problem Hamiltonian Cycle Problem. It is a

well-known NP-Complete problem

1. The first step is to prove that Hamiltonian circuit is a NP-Complete problem. One can
guess many sequences of cities as certificates. The verification algorithm is possible if a tour can
be guessed which is of length of k. This verification can be done in polynomial time. Therefore,
Hamiltonian cycle is a NP-Complete problem.
2. In step 2, Hamiltonian cycle can be reduced to Traveling Salesman problem in polynomial
time. This is done by taking arbitrary G instance of Hamiltonian cycle and construct G’ and bound
k such that G has Hamilton cycle iff G’ has a tour of length k

Page 9
 UNIT 5

Assign k =n. the graph G has a Hamiltonian cycle iff G’ has a tour of cost at most 0. If the
graph G has a Hamiltonian cycle h, then all the edges belongs to E and thus has a cost of at
most 0 in G’. Thus h is a tour in G’ with cost of 0.

Conversely, suppose graph G’ has a tour h’ of cost 0. Since the cost of the tour is 0, each
edge of the tour must have cost of 0. Therefore, h’ contains only edges in E. Therefore, one can
conclude that h’ is a Hamiltonian cycle in graph G [3]. Therefore, one can conclude that TSP is
as hard as Hamiltonian cycle.

Clique problem

The NP-Complete proof for Clique problem can be give as per the proof outline. Let us
formally, state the clique problem as follows:

Given a Graph G = (V,E) and a positive integer k, Does G contain a clique of size k? Clique k is
a complete subgraph of graph G on k vertices.

For example, for the sample graph shown in Fig. 1, the clique is shown in Fig. 2.

It can be observed the subgraph of three vertices shown in Fig. 2. Is the clique for the sample
graph shown in Fig. 1.

1. In step 1, a well-known NP-Complete problem, SAT (Formula Satisfiability problem) is

chosen. One can recollect from module 34, Satisfiability problem can be given as
follows:

Page 10
 UNIT 5

Given a Boolean formula, determine whether this formula is satisfiable or not. The Boolean

formula may have a literal : x1 or x1 , a clause C like x1 v x2 v x3 and formula be in the

conjunctive normal form as C1& C2 & … & Cm . It can be observed that there would be m-
clauses.

The reduction from SAT problem to clique problem is given as follows:

Construct a graph G = (V,E) where V = 2n literals and edge as

2. In step 2, one can observe that F is satisfiable iff G has a clique of size m
3. In step 3, one can argue that a clique of m corresponds to assignment of true to m
literals in m different clauses. It should also be observed that

• An edge is between only non-contradictory nodes. Hence, f is satisfiable iff there is non-
contradictory assignment of true to m literals.
• This is possible iff G has a clique of size m.

Therefore, one can conclude that Clique problem is NP-Complete. Page 11

 UNIT 5

Vertex Cover

Vertex cover is discussed in module 32 and module 33. One can recollect that the vertex
cover problem can be stated as follows:
Given a graph G = (V,E) and a positive integer k, Find a subset C of size k such that each edge
in E is incident to at least one vertex in C

Solution

No. Because, Edges (4, 5), (5, 6) and (3, 6) are not covered by it . So, the idea is to cover all
edges of the given graph, by picking the extra vertices 4 and 3 so that all edges are covered. The
idea is thus to pick a minimum set of vertices so as to cover the graph. The proof for vertex cover
can be given based on [4] as follows: First a Formula SAT is chosen and the graph is constructed
using the following rules.

Page 12
 UNIT 5

Thus one can conclude that vertex cover proble m is NP-Complete.

Summary

One can conclude from this module 35 that

• NP-Complete proof can be given by reducing a well known NP-Complete problem to the
given problem. Page 13
 UNIT 5
• Traveling Salesman is proven NP complete by reducing Hamiltonian problem to TSP
• Clique problem is NP-Complete
• Vertex Cover problem is NP-Complete

Complexity Classes
Definition of NP class Problem: - The set of all decision-based problems came into the division
of NP Problems who can't be solved or produced an output within polynomial time but verified in
the polynomial time. NP class contains P class as a subset. NP problems being hard to solve.

Note: - The term "NP" does not mean "not polynomial." Originally, the term meant "non-deterministic
polynomial. It means according to the one input number of output will be produced.

Definition of P class Problem: - The set of decision-based problems come into the division of P
Problems who can be solved or produced an output within polynomial time. P problems being
easy to solve

Definition of Polynomial time: - If we produce an output according to the given input within a
specific amount of time such as within a minute, hours. This is known as Polynomial time.

Definition of Non-Polynomial time: - If we produce an output according to the given input but
there are no time constraints is known as Non-Polynomial time. But yes output will produce but
time is not fixed yet.

Skip Ad

Definition of Decision Based Problem: - A problem is called a decision problem if its output is a
simple "yes" or "no" (or you may need this of this as true/false, 0/1, accept/reject.) We will phrase
many optimization problems as decision problems. For example, Greedy method, D.P., given a
graph G= (V, E) if there exists any Hamiltonian cycle.

Definition of NP-hard class: - Here you to satisfy the following points to come into the division
of NP-hard

1. If we can solve this problem in polynomial time, then we can solve all NP problems in
polynomial time
2. If you convert the issue into one form to another form within the polynomial time

Definition of NP-complete class: - A problem is in NP-complete, if

1. It is in NP
2. It is NP-hard

Pictorial representation of all NP classes which includes NP, NP-hard, and NP-complete

Page 14
 UNIT 5

Fig: Complexity Classes

Polynomial Time Verification

Before talking about the class of NP-complete problems, it is essential to introduce the notion of a
verification algorithm.

Many problems are hard to solve, but they have the property that it easy to authenticate the
solution if one is provided.

Hamiltonian cycle problem: -

Consider the Hamiltonian cycle problem. Given an undirected graph G, does G have a cycle that
visits each vertex exactly once? There is no known polynomial time algorithm for this dispute.

Fig: Hamiltonian Cycle

Skip Ad

Let us understand that a graph did have a Hamiltonian cycle. It would be easy for someone to
convince of this. They would similarly say: "the period is hv3, v7, v1....v13i.
Page 15
 UNIT 5
We could then inspect the graph and check that this is indeed a legal cycle and that it visits all of
the vertices of the graph exactly once. Thus, even though we know of no efficient way to solve the
Hamiltonian cycle problem, there is a beneficial way to verify that a given cycle is indeed a
Hamiltonian cycle.

Note:-For the verification in the Polynomial-time of an undirected Hamiltonian cycle graph G. There
must be exact/specific/definite path must be given of Hamiltonian cycle then you can verify in the
polynomial time.

Definition of Certificate: - A piece of information which contains in the given path of a vertex is
known as certificate

Relation of P and NP classes

1. P contains in NP
2. P=NP

1. Observe that P contains in NP. In other words, if we can solve a problem in polynomial time,
we can indeed verify the solution in polynomial time. More formally, we do not need to see
a certificate (there is no need to specify the vertex/intermediate of the specific path) to
solve the problem; we can explain it in polynomial time anyway.
2. However, it is not known whether P = NP. It seems you can verify and produce an output of
the set of decision-based problems in NP classes in a polynomial time which is impossible
because according to the definition of NP classes you can verify the solution within the
polynomial time. So this relation can never be held.

Reductions:
The class NP-complete (NPC) problems consist of a set of decision problems (a subset of class NP)
that no one knows how to solve efficiently. But if there were a polynomial solution for even a
single NP-complete problem, then every problem in NPC will be solvable in polynomial time. For
this, we need the concept of reductions.

Suppose there are two problems, A and B. You know that it is impossible to solve problem A in
polynomial time. You want to prove that B cannot be explained in polynomial time. We want to
show that (A ∉ P) => (B ∉ P)

Consider an example to illustrate reduction: The following problem is well-known to be NPC:

3-color: Given a graph G, can each of its vertices be labeled with one of 3 different colors such
that two adjacent vertices do not have the same label (color).

Coloring arises in various partitioning issues where there is a constraint that two objects cannot be
assigned to the same set of partitions. The phrase "coloring" comes from the original application
which was in map drawing. Two countries that contribute a common border should be colored
with different colors.

Page 16
 UNIT 5
It is well known that planar graphs can be colored (maps) with four colors. There exists a
polynomial time algorithm for this. But deciding whether this can be done with 3 colors is hard,
and there is no polynomial time algorithm for it.

Fig: Example of 3-colorable and non-3-colorable graphs.

Polynomial Time Reduction:

We say that Decision Problem L1 is Polynomial time Reducible to decision Problem L2 (L1≤p L2) if
there is a polynomial time computation function f such that of all x, xϵL1 if and only if xϵL2

NP-Completeness
A decision problem L is NP-Hard if

L' ≤p L for all L' ϵ NP.

Definition: L is NP-complete if

1. L ϵ NP and
2. L' ≤ p L for some known NP-complete problem L.' Given this formal definition, the
complexity classes are:

P: is the set of decision problems that are solvable in polynomial time.

Skip Ad

NP: is the set of decision problems that can be verified in polynomial time.

NP-Hard: L is NP-hard if for all L' ϵ NP, L' ≤p L. Thus if we can solve L in polynomial time, we can
solve all NP problems in polynomial time.

NP-Complete L is NP-complete if

1. L ϵ NP and
Page 17
 UNIT 5
2. L is NP-hard

If any NP-complete problem is solvable in polynomial time, then every NP-Complete problem is
also solvable in polynomial time. Conversely, if we can prove that any NP-Complete problem
cannot be solved in polynomial time, every NP-Complete problem cannot be solvable in
polynomial time.

Reductions
Concept: - If the solution of NPC problem does not exist then the conversion from one NPC
problem to another NPC problem within the polynomial time. For this, you need the concept of
reduction. If a solution of the one NPC problem exists within the polynomial time, then the rest of
the problem can also give the solution in polynomial time (but it's hard to believe). For this, you
need the concept of reduction.

Example: - Suppose there are two problems, A and B. You know that it is impossible to solve
problem A in polynomial time. You want to prove that B cannot be solved in polynomial time. So
you can convert the problem A into problem B in polynomial time.

Example of NP-Complete problem

NP problem: - Suppose a DECISION-BASED problem is provided in which a set of inputs/high
inputs you can get high output.

Criteria to come either in NP-hard or NP-complete.

1. The point to be noted here, the output is already given, and you can verify the
output/solution within the polynomial time but can't produce an output/solution in
polynomial time.
2. Here we need the concept of reduction because when you can't produce an output of the
problem according to the given input then in case you have to use an emphasis on the
concept of reduction in which you can convert one problem into another problem.

Note1:- If you satisfy both points then your problem comes into the category of NP-complete class

Note2:- If you satisfy the only 2nd points then your problem comes into the category of NP-hard class

So according to the given decision-based NP problem, you can decide in the form of yes or no. If,
yes then you have to do verify and convert into another problem via reduction concept. If you are
being performed, both then decision-based NP problems are in NP compete.

Here we will emphasize NPC.

Page 18
 UNIT 5

CIRCUIT SAT
According to given decision-based NP problem, you can design the CIRCUIT and verify a given
mentioned output also within the P time. The CIRCUIT is provided below:-

Note:- You can design a circuit and verified the mentioned output within Polynomial time but remember
you can never predict the number of gates which produces the high output against the set of
inputs/high inputs within a polynomial time. So you verified the production and conversion had been
done within polynomial time. So you are in NPC.

SAT (Satisfiability):-
A Boolean function is said to be SAT if the output for the given value of the input is true/high/1

F=X+YZ (Created a Boolean function by CIRCUIT SAT)

These points you have to be performed for NPC

Keep Watching

1. CONCEPTS OF SAT
2. CIRCUIT SAT≤ρ SAT
3. SAT≤ρ CIRCUIT SAT
4. SAT ϵ NPC

1. CONCEPT: - A Boolean function is said to be SAT if the output for the given value of the
input is true/high/1.
2. CIRCUIT SAT≤ρ SAT: - In this conversion, you have to convert CIRCUIT SAT into SAT within
the polynomial time as we did it Page 19
 UNIT 5
3. SAT≤ρ CIRCUIT SAT: - For the sake of verification of an output you have to convert SAT
into CIRCUIT SAT within the polynomial time, and through the CIRCUIT SAT you can get the
verification of an output successfully
4. SAT ϵ NPC: - As you know very well, you can get the SAT through CIRCUIT SAT that comes
from NP.

Proof of NPC: - Reduction has been successfully made within the polynomial time from CIRCUIT
SAT TO SAT. Output has also been verified within the polynomial time as you did in the above
conversation.

So concluded that SAT ϵ NPC.

3CNF SAT
Concept: - In 3CNF SAT, you have at least 3 clauses, and in clauses, you will have almost 3 literals
or constants

Such as (X+Y+Z) (X+Y+Z) (X+Y+Z)

You can define as (XvYvZ) ᶺ (XvYvZ) ᶺ (XvYvZ)
V=OR operator
^ =AND operator

These all the following points need to be considered in 3CNF SAT.

To prove: -

1. Concept of 3CNF SAT

2. SAT≤ρ 3CNF SAT
3. 3CNF≤ρ SAT
4. 3CNF ϵ NPC

1. CONCEPT: - In 3CNF SAT, you have at least 3 clauses, and in clauses, you will have almost 3
literals or constants.
2. SAT ≤ρ 3CNF SAT:- In which firstly you need to convert a Boolean function created in SAT
into 3CNF either in POS or SOP form within the polynomial time
F=X+YZ
= (X+Y) (X+Z)
= (X+Y+ZZ') (X+YY'+Z)
= (X+Y+Z) (X+Y+Z') (X+Y+Z) (X+Y'+Z)
= (X+Y+Z) (X+Y+Z') (X+Y'+Z)
3. 3CNF ≤p SAT: - From the Boolean Function having three literals we can reduce the whole
function into a shorter one.
F= (X+Y+Z) (X+Y+Z') (X+Y'+Z)
= (X+Y+Z) (X+Y+Z') (X+Y+Z) (X+Y'+Z)
= (X+Y+ZZ') (X+YY'+Z) Page 20
 UNIT 5
= (X+Y) (X+Z)
= X+YZ
4. 3CNF ϵ NPC: - As you know very well, you can get the 3CNF through SAT and SAT through
CIRCUIT SAT that comes from NP.

Proof of NPC:-

1. It shows that you can easily convert a Boolean function of SAT into 3CNF SAT and satisfied
the concept of 3CNF SAT also within polynomial time through Reduction concept.
2. If you want to verify the output in 3CNF SAT then perform the Reduction and convert into
SAT and CIRCUIT also to check the output

If you can achieve these two points that means 3CNF SAT also in NPC

Clique
To Prove: - Clique is an NPC or not?

For this you have to satisfy the following below-mentioned points: -

1. Clique
2. 3CNF ≤ρ Clique
3. Clique ≤ρ 3CNF≤SAT
4. Clique ϵ NP

1) Clique
Definition: - In Clique, every vertex is directly connected to another vertex, and the number of
vertices in the Clique represents the Size of Clique.

CLIQUE COVER: - Given a graph G and an integer k, can we find k subsets of verticesV1, V2...VK,
such that UiVi = V, and that each Vi is a clique of G.3

The following figure shows a graph that has a clique cover of size 3.

2)3CNF ≤ρ Clique
Page 21
 UNIT 5
Proof:-For the successful conversion from 3CNF to Clique, you have to follow the two steps:-

Draw the clause in the form of vertices, and each vertex represents the literals of the clauses.

1. They do not complement each other

2. They don't belong to the same clause
In the conversion, the size of the Clique and size of 3CNF must be the same, and you
successfully converted 3CNF into Clique within the polynomial time

Clique ≤ρ 3CNF
Proof: - As you know that a function of K clause, there must exist a Clique of size k. It means that P
variables which are from the different clauses can assign the same value (say it is 1). By using these
values of all the variables of the CLIQUES, you can make the value of each clause in the function is
equal to 1

Example: - You have a Boolean function in 3CNF:-

(X+Y+Z) (X+Y+Z') (X+Y'+Z)

After Reduction/Conversion from 3CNF to CLIQUE, you will get P variables such as: - x +y=1, x
+z=1 and x=1

Put the value of P variables in equation (i)

(1+1+0)(1+0+0)(1+0+1)

(1)(1)(1)=1 output verified

4) Clique ϵ NP:-
Proof: - As you know very well, you can get the Clique through 3CNF and to convert the decision-
based NP problem into 3CNF you have to first convert into SAT and SAT comes from NP.

So, concluded that CLIQUE belongs to NP.

Proof of NPC:-

1. Reduction achieved within the polynomial time from 3CNF to Clique

2. And verified the output after Reduction from Clique To 3CNF above
So, concluded that, if both Reduction and verification can be done within the polynomial
time that means Clique also in NPC.

Vertex Cover
1. Vertex Cover Definition
2. Vertex Cover ≤ρ Clique
3. Clique ≤ρ Vertex Cover
Page 22
 UNIT 5
4. Vertex Cover ϵ NP

1) Vertex Cover:
Definition: - It represents a set of vertex or node in a graph G (V, E), which gives the connectivity
of a complete graph

According to the graph G of vertex cover which you have created, the size of Vertex Cover =2

2) Vertex Cover ≤ρ Clique

In a graph G of Vertex Cover, you have N vertices which contain a Vertex Cover K. There must exist
of Clique Size of size N-K in its complement.

According to the graph G, you have

Number of vertices=6
Size of Clique=N-K=4

You can also create the Clique by complimenting the graph G of Vertex Cover means in simpler
form connect the vertices in Vertex Cover graph G through edges where edges don?t exist and
remove all the existed edges

You will get the graph G with Clique Size=4

3) Clique ≤ρ Vertex Cover

Here through the Reduction process, you can get the Vertex Cover form Clique by just
complimenting the Clique graph G within the polynomial time.

4) Vertex Cover ϵ NP
As you know very well, you can get the Vertex Cover through Clique and to convert the decision-
based NP problem into Clique firstly you have to convert into 3CNF and 3CNF into SAT and SAT
into CIRCUIT SAT that comes from NP.

Proof of NPC:-

1. Reduction from Clique to Vertex Cover has been made within the polynomial time. In the
simpler form, you can convert into Vertex Cover from Clique within the polynomial time
Page 23
 UNIT 5
2. And verification has also been done when you convert Vertex Cover to Clique and Clique to
3CNF and satisfy/verified the output within a polynomial time also, so it concluded that
Reduction and Verification had been done in the polynomial time that means Vertex Cover
also comes in NPC

Subset Cover
To Prove:-

1. Subset Cover
2. Vertex Cover ≤ρ Subset Cover
3. Subset Cover≤ρ Vertex Cover
4. Subset Cover ϵ NP

1) Subset Cover
Definition: - Number of a subset of edges after making the union for a get all the edges of the
complete graph G, and that is called Subset Cover.

According to the graph G, which you have created the size of Subset Cover=2

1. v1{e1,e6} v2{e5,e2} v3{e2,e4,e6} v4{e1,e3,e5} v5{e4} v6{e3}

2. v3Uv4= {e1, e2, e3, e4, e5, e6} complete set of edges after the union of vertices.

2) Vertex Cover ≤ρ Subset Cover

In a graph G of vertices N, if there exists a Vertex Cover of size k, then there must also exist a
Subset Cover of size k even. If you can achieve after the Reduction from Vertex Cover to Subset
Cover within a polynomial time, which means you did right.

Keep Watching

3) Subset Cover ≤ρ Vertex Cover

Page 24
 UNIT 5
Just for verification of the output perform the Reduction and create Clique and via an equation, N-
K verifies the Clique also and through Clique you can quickly generate 3CNF and after solving the
Boolean function of 3CNF in the polynomial time. You will get output. It means the output has
been verified.

4) Subset Cover ϵ NP:-

Proof: - As you know very well, you can get the Subset-Cover through Vertex Cover and Vertex
Cover through Clique and to convert the decision-based NP problem into Clique firstly you have to
convert into3CNF and 3CNF into SAT and SAT into CIRCUIT SAT that comes from NP.

Proof of NPC:-

The Reduction has been successfully made within the polynomial time form Vertex Cover to Subset
Cover

Output has also been verified within the polynomial time as you did in the above conversation so,
concluded that SUBSET COVER also comes in NPC.

Independent Set:
An independent set of a graph G = (V, E) is a subset V'⊆V of vertices such that every edge in E is
incident on at most one vertex in V.' The independent-set problem is to find a largest-size
independent set in G. It is not hard to find small independent sets, e.g., a small independent set is
an individual node, but it is hard to find large independent sets.

Approximate Algorithms
Introduction:
An Approximate Algorithm is a way of approach NP-COMPLETENESS for the optimization
problem. This technique does not guarantee the best solution. The goal of an approximation
algorithm is to come as close as possible to the optimum value in a reasonable amount of time
which is at the most polynomial time. Such algorithms are called approximation algorithm or
heuristic algorithm.

o For the traveling salesperson problem, the optimization problem is to find the shortest
cycle, and the approximation problem is to find a short cycle.
Page 25
 UNIT 5
o For the vertex cover problem, the optimization problem is to find the vertex cover with
fewest vertices, and the approximation problem is to find the vertex cover with few vertices.

Performance Ratios
Suppose we work on an optimization problem where every solution carries a cost. An Approximate
Algorithm returns a legal solution, but the cost of that legal solution may not be optimal.

For Example, suppose we are considering for a minimum size vertex-cover (VC). An
approximate algorithm returns a VC for us, but the size (cost) may not be minimized.

Another Example is we are considering for a maximum size Independent set (IS). An
approximate Algorithm returns an IS for us, but the size (cost) may not be maximum. Let C be the
cost of the solution returned by an approximate algorithm, and C* is the cost of the optimal
solution.5

We say the approximate algorithm has an approximate ratio P (n) for an input size n, where

Intuitively, the approximation ratio measures how bad the approximate solution is distinguished
with the optimal solution. A large (small) approximation ratio measures the solution is much worse
than (more or less the same as) an optimal solution.

Observe that P (n) is always ≥ 1, if the ratio does not depend on n, we may write P. Therefore, a
1-approximation algorithm gives an optimal solution. Some problems have polynomial-time
approximation algorithm with small constant approximate ratios, while others have best-known
polynomial time approximation algorithms whose approximate ratios grow with n.

Vertex Cover
A Vertex Cover of a graph G is a set of vertices such that each edge in G is incident to at least one
of these vertices.

The decision vertex-cover problem was proven NPC. Now, we want to solve the optimal version of
the vertex cover problem, i.e., we want to find a minimum size vertex cover of a given graph. We
call such vertex cover an optimal vertex cover C*.

An approximate algorithm for vertex cover:

Page 26
 UNIT 5
Approx-Vertex-Cover (G = (V, E))
{
C = empty-set;
E'= E;
While E' is not empty do
{
Let (u, v) be any edge in E': (*)
Add u and v to C;
Remove from E' all edges incident to
u or v;
}
Return C;
}

The idea is to take an edge (u, v) one by one, put both vertices to C, and remove all the edges
incident to u or v. We carry on until all edges have been removed. C is a VC. But how good is C?

VC = {b, c, d, e, f, g}

Traveling-salesman Problem
In the traveling salesman Problem, a salesman must visits n cities. We can say that salesman wishes
to make a tour or Hamiltonian cycle, visiting each city exactly once and finishing at the city he
starts from. There is a non-negative cost c (i, j) to travel from the city i to city j. The goal is to find a
tour of minimum cost. We assume that every two cities are connected. Such problems are called
Traveling-salesman problem (TSP).

We can model the cities as a complete graph of n vertices, where each vertex represents a city.

It can be shown that TSP is NPC.

Page 27
 UNIT 5
If we assume the cost function c satisfies the triangle inequality, then we can use the following
approximate algorithm.

Triangle inequality
Let u, v, w be any three vertices, we have

One important observation to develop an approximate solution is if we remove an edge from H*,
the tour becomes a spanning tree.

Approx-TSP (G= (V, E))

{
1. Compute a MST T of G;
2. Select any vertex r is the root of the tree;
3. Let L be the list of vertices visited in a preorder tree walk of T;
4. Return the Hamiltonian cycle H that visits the vertices in the order L;
}

Traveling-salesman Problem

Intuitively, Approx-TSP first makes a full walk of MST T, which visits each edge exactly two times.
To create a Hamiltonian cycle from the full walk, it bypasses some vertices (which corresponds to
making a shortcut

Page 28
 UNIT 5

The Naive String Matching Algorithm

The naïve approach tests all the possible placement of Pattern P [1.......m] relative to text T [1......n].
We try shift s = 0, 1.......n-m, successively and for each shift s. Compare T [s+1.......s+m] to P [1......m].

The naïve algorithm finds all valid shifts using a loop that checks the condition P [1.......m] = T
[s+1.......s+m] for each of the n - m +1 possible value of s.

NAIVE-STRING-MATCHER (T, P)
1. n ← length [T]
2. m ← length [P]
3. for s ← 0 to n -m
4. do if P [1.....m] = T [s + 1....s + m]
5. then print "Pattern occurs with shift" s

Analysis: This for loop from 3 to 5 executes for n-m + 1(we need at least m characters at the end)
times and in iteration we are doing m comparisons. So the total complexity is O (n-m+1).

Example:

Suppose T = 1011101110
P = 111
Find all the Valid Shift

Solution:3

Page 29
 UNIT 5

The Rabin-Karp-Algorithm
The Rabin-Karp string matching algorithm calculates a hash value for the pattern, as well as for
each M-character subsequences of text to be compared. If the hash values are unequal, the
algorithm will determine the hash value for next M-character sequence. If the hash values are
equal, the algorithm will analyze the pattern and the M-character sequence. In this way, there is
only one comparison per text subsequence, and character matching is only required when the
hash values match.

RABIN-KARP-MATCHER (T, P, d, q)
1. n ← length [T]
2. m ← length [P]
3. h ← dm-1 mod q
4. p ← 0
5. t0 ← 0 Page 30
 UNIT 5
6. for i ← 1 to m
7. do p ← (dp + P[i]) mod q
8. t0 ← (dt0+T [i]) mod q
9. for s ← 0 to n-m
10. do if p = ts
11. then if P [1.....m] = T [s+1.....s + m]
12. then "Pattern occurs with shift" s
13. If s < n-m
14. then ts+1 ← (d (ts-T [s+1]h)+T [s+m+1])mod q

Example: For string matching, working module q = 11, how many spurious hits does the Rabin-
Karp matcher encounters in Text T = 31415926535.......

T = 31415926535.......
P = 26
Here T.Length =11 so Q = 11
And P mod Q = 26 mod 11 = 4
Now find the exact match of P mod Q...

Solution:

Page 31
 UNIT 5

Page 32
 UNIT 5
Complexity:
The running time of RABIN-KARP-MATCHER in the worst case scenario O ((n-m+1) m but it has
a good average case running time. If the expected number of strong shifts is small O (1) and prime
q is chosen to be quite large, then the Rabin-Karp algorithm can be expected to run in time O
(n+m) plus the time to require to process spurious hits.

String Matching with Finite Automata

The string-matching automaton is a very useful tool which is used in string matching algorithm. It
examines every character in the text exactly once and reports all the valid shifts in O (n) time. The
goal of string matching is to find the location of specific text pattern within the larger body of text
(a sentence, a paragraph, a book, etc.)

Finite Automata:
A finite automaton M is a 5-tuple (Q, q0,A,∑δ), where

o Q is a finite set of states,

o q0 ∈ Q is the start state,
o A ⊆ Q is a notable set of accepting states,
o ∑ is a finite input alphabet,
o δ is a function from Q x ∑ into Q called the transition function of M.

The finite automaton starts in state q0 and reads the characters of its input string one at a time. If
the automaton is in state q and reads input character a, it moves from state q to state δ (q, a).
Whenever its current state q is a member of A, the machine M has accepted the string read so far.
An input that is not allowed is rejected.

A finite automaton M induces a function ∅ called the called the final-state function, from ∑* to Q
such that ∅(w) is the state M ends up in after scanning the string w. Thus, M accepts a string w if
and only if ∅(w) ∈ A.

The function f is defined as

∅ (∈)=q0
∅ (wa) = δ ((∅ (w), a) for w ∈ ∑*,a∈ ∑)
FINITE- AUTOMATON-MATCHER (T,δ, m),
1. n ← length [T]
2. q ← 0
3. for i ← 1 to n
4. do q ← δ (q, T[i])
5. If q =m
6. then s←i-m
7. print "Pattern occurs with shift s" s

Page 33
 UNIT 5
The primary loop structure of FINITE- AUTOMATON-MATCHER implies that its running time on a
text string of length n is O (n).

Computing the Transition Function: The following procedure computes the transition function δ
from given pattern P [1......m]

COMPUTE-TRANSITION-FUNCTION (P, ∑)
1. m ← length [P]
2. for q ← 0 to m
3. do for each character a ∈ ∑*
4. do k ← min (m+1, q+2)
5. repeat k←k-1
6. Until
7. δ(q,a)←k
8. Return δ

Example: Suppose a finite automaton which accepts even number of a's where ∑ = {a, b, c}

Solution:

q0 is the initial state.

Page 34
 UNIT 5

The Knuth-Morris-Pratt (KMP)Algorithm

Knuth-Morris and Pratt introduce a linear time algorithm for the string matching problem. A
matching time of O (n) is achieved by avoiding comparison with an element of 'S' that have
previously been involved in comparison with some element of the pattern 'p' to be matched. i.e.,
backtracking on the string 'S' never occurs

Components of KMP Algorithm:

1. The Prefix Function (Π): The Prefix Function, Π for a pattern encapsulates knowledge about
how the pattern matches against the shift of itself. This information can be used to avoid a useless
shift of the pattern 'p.' In other words, this enables avoiding backtracking of the string 'S.'

2. The KMP Matcher: With string 'S,' pattern 'p' and prefix function 'Π' as inputs, find the
occurrence of 'p' in 'S' and returns the number of shifts of 'p' after which occurrences are found.

The Prefix Function (Π)

Following pseudo code compute the prefix function, Π:

Skip Ad

COMPUTE- PREFIX- FUNCTION (P)

1. m ←length [P] //'p' pattern to be matched
2. Π [1] ← 0
3. k ← 0
4. for q ← 2 to m
5. do while k > 0 and P [k + 1] ≠ P [q]
6. do k ← Π [k]
7. If P [k + 1] = P [q]
8. then k← k + 1
9. Π [q] ← k
10. Return Π

Running Time Analysis:

In the above pseudo code for calculating the prefix function, the for loop from step 4 to step 10
runs 'm' times. Step1 to Step3 take constant time. Hence the running time of computing prefix
function is O (m).

Example: Compute Π for the pattern 'p' below:

Solution:
Initially: m = length [p] = 7
Π [1] = 0
k = 0

Page 35
 UNIT 5

Page 36
 UNIT 5
After iteration 6 times, the prefix function computation is complete:

The KMP Matcher:

The KMP Matcher with the pattern 'p,' the string 'S' and prefix function 'Π' as input, finds a match
of p in S. Following pseudo code compute the matching component of KMP algorithm:

KMP-MATCHER (T, P)
1. n ← length [T]
2. m ← length [P]
3. Π← COMPUTE-PREFIX-FUNCTION (P)
4. q ← 0 // numbers of characters matched
5. for i ← 1 to n // scan S from left to right
6. do while q > 0 and P [q + 1] ≠ T [i]
7. do q ← Π [q] // next character does not match
8. If P [q + 1] = T [i]
9. then q ← q + 1 // next character matches
10. If q = m // is all of p matched?
11. then print "Pattern occurs with shift" i - m
12. q ← Π [q] // look for the next match

Running Time Analysis:

The for loop beginning in step 5 runs 'n' times, i.e., as long as the length of the string 'S.' Since step
1 to step 4 take constant times, the running time is dominated by this for the loop. Thus running
time of the matching function is O (n).

Example: Given a string 'T' and pattern 'P' as follows:

Let us execute the KMP Algorithm to find whether 'P' occurs in 'T.'

For 'p' the prefix function, ? was computed previously and is as follows:

Solution:

Initially: n = size of T = 15
m = size of P = 7
Page 37
 UNIT 5

Page 38
 UNIT 5

Page 39
 UNIT 5

Pattern 'P' has been found to complexity occur in a string 'T.' The total number of shifts that took
place for the match to be found is i-m = 13 - 7 = 6 shifts

The Boyer-Moore Algorithm

Robert Boyer and J Strother Moore established it in 1977. The B-M String search algorithm is a
particularly efficient algorithm and has served as a standard benchmark for string search algorithm
ever since.

The B-M algorithm takes a 'backward' approach: the pattern string (P) is aligned with the start of
the text string (T), and then compares the characters of a pattern from right to left, beginning with
rightmost character.

Page 40
 UNIT 5
If a character is compared that is not within the pattern, no match can be found by analyzing any
further aspects at this position so the pattern can be changed entirely past the mismatching
character.

For deciding the possible shifts, B-M algorithm uses two preprocessing strategies simultaneously.
Whenever a mismatch occurs, the algorithm calculates a variation using both approaches and
selects the more significant shift thus, if make use of the most effective strategy for each case.

The two strategies are called heuristics of B - M as they are used to reduce the search. They are:

1. Bad Character Heuristics

2. Good Suffix Heuristics

1. Bad Character Heuristics

This Heuristics has two implications:

o Suppose there is a character in a text in which does not occur in a pattern at all. When a
mismatch happens at this character (called as bad character), the whole pattern can be
changed, begin matching form substring next to this 'bad character.'
o On the other hand, it might be that a bad character is present in the pattern, in this case,
align the nature of the pattern with a bad character in the text.

Thus in any case shift may be higher than one.

Example1: Let Text T = <nyoo nyoo> and pattern P = <noyo>

Example2: If a bad character doesn't exist the pattern then.

Page 41
 UNIT 5

Problem in Bad-Character Heuristics:

In some cases, Bad-Character Heuristics produces some negative shifts.

For Example:

This means that we need some extra information to produce a shift on encountering a bad
character. This information is about the last position of every aspect in the pattern and also the set
of characters used in a pattern (often called the alphabet ∑of a pattern).

COMPUTE-LAST-OCCURRENCE-FUNCTION (P, m, ∑ )
1. for each character a ∈ ∑
2. do λ [a] = 0
3. for j ← 1 to m
4. do λ [P [j]] ← j
5. Return λ

2. Good Suffix Heuristics:

A good suffix is a suffix that has matched successfully. After a mismatch which has a negative shift
in bad character heuristics, look if a substring of pattern matched till bad character has a good
suffix in it, if it is so then we have an onward jump equal to the length of suffix found.

Example:

Page 42
 UNIT 5

COMPUTE-GOOD-SUFFIX-FUNCTION (P, m)
1. Π ← COMPUTE-PREFIX-FUNCTION (P)
2. P'← reverse (P)
3. Π'← COMPUTE-PREFIX-FUNCTION (P')
4. for j ← 0 to m
5. do ɣ [j] ← m - Π [m]
6. for l ← 1 to m
7. do j ← m - Π' [L]
8. If ɣ [j] > l - Π' [L]
9. then ɣ [j] ← 1 - Π'[L]
10. Return ɣ

BOYER-MOORE-MATCHER (T, P, ∑)
1. n ←length [T]
2. m ←length [P]
3. λ← COMPUTE-LAST-OCCURRENCE-FUNCTION (P, m, ∑ )
4. ɣ← COMPUTE-GOOD-SUFFIX-FUNCTION (P, m)
5. s ←0
6. While s ≤ n - m
7. do j ← m
8. While j > 0 and P [j] = T [s + j]
9. do j ←j-1
10. If j = 0
11. then print "Pattern occurs at shift" s
12. s ← s + ɣ[0]
13. else s ← s + max (ɣ [j], j - λ[T[s+j]])

Complexity Comparison of String Matching Algorithm:

Algorithm Preprocessing Time Matching Time

Naive O (O (n - m + 1)m)

Rabin-Karp O(m) (O (n - m + 1)m)

Finite Automata O(m|∑|) O (n)

Knuth-Morris-Pratt O(m) O (n)

Boyer-Moore O(|∑|) (O ((n - m + 1) + |∑|))

Page 43