Mc4101 Ads Notes Advance Data Structure Nodes

Download as pdf or txt
Download as pdf or txt
You are on page 1of 144

lOMoARcPSD|33989822

MC4101 - ADS Notes - Advance data structure nodes

Master of Computer Applications (Anna University)

Studocu is not sponsored or endorsed by any college or university


Downloaded by Raj Kumari ([email protected])
lOMoARcPSD|33989822

MC4101 ADVANCED DATA STRUCTURES AND ALGORITHMS LTPC


3 003
COURSE OBJECTIVES:
 To understand the usage of algorithms in computing
 To learn and use hierarchical data structures and its operations
 To learn the usage of graphs and its applications
 To select and design data structures and algorithms that is appropriate for problems
 To study about NP Completeness of problems.

UNIT I ROLE OF ALGORITHMS IN COMPUTING & COMPLEXITY 9


ANALYSIS
Algorithms – Algorithms as a Technology -Time and Space complexity of algorithms- Asymptotic analysis-
Average and worst-case analysis-Asymptotic notation-Importance of efficient algorithms- Program
performance measurement - Recurrences: The Substitution Method – The Recursion- Tree Method- Data
structures and algorithms.

UNIT II HIERARCHICAL DATA STRUCTURES 9


Binary Search Trees: Basics – Querying a Binary search tree – Insertion and Deletion- Red Black trees:
Properties of Red-Black Trees – Rotations – Insertion – Deletion -B-Trees: Definition of B - trees – Basic
operations on B-Trees – Deleting a key from a B-Tree- Heap – Heap Implementation – Disjoint Sets -
Fibonacci Heaps: structure – Mergeable-heap operations- Decreasing a key and deleting a node-Bounding
the maximum degree.

UNIT III GRAPHS 9


Elementary Graph Algorithms: Representations of Graphs – Breadth-First Search – Depth-First Search –
Topological Sort – Strongly Connected Components- Minimum Spanning Trees: Growing a Minimum
Spanning Tree – Kruskal and Prim- Single-Source Shortest Paths: The Bellman-Ford algorithm – Single-
Source Shortest paths in Directed Acyclic Graphs – Dijkstra‘s Algorithm; Dynamic Programming - All-
Pairs Shortest Paths: Shortest Paths and Matrix Multiplication – The Floyd-Warshall Algorithm

UNIT IV ALGORITHM DESIGN TECHNIQUES 9


Dynamic Programming: Matrix-Chain Multiplication – Elements of Dynamic Programming – Longest
Common Subsequence- Greedy Algorithms: – Elements of the Greedy Strategy- An Activity-Selection
Problem - Huffman Coding.

UNIT V NP COMPLETE AND NP HARD 9


NP-Completeness: Polynomial Time – Polynomial-Time Verification – NP- Completeness and
Reducibility – NP-Completeness Proofs – NP-Complete Problems.
TOTAL : 45 PERIODS

SUGGESTED ACTIVITIES:
1. Write an algorithm for Towers of Hanoi problem using recursion and analyze the

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

complexity (No of disc-4)


2. Write any one real time application of hierarchical data structure
3. Write a program to implement Make_Set, Find_Set and Union functions for Disjoint Set Data
Structure for a given undirected graph G(V,E) using the linked list representation with simple
implementation of Union operation
4. Find the minimum cost to reach last cell of the matrix from its first cell
5. Discuss about any NP completeness problem

COURSE OUTCOMES:
CO1:Design data structures and algorithms to solve computing problems.
CO2:Choose and implement efficient data structures and apply them to solve problems. CO3:Design
algorithms using graph structure and various string-matching algorithms tosolve real-life
problems.
CO4: Design one’s own algorithm for an unknown problem.
CO5: Apply suitable design strategy for problem solving.

REFERENCES
1. S.Sridhar,” Design and Analysis of Algorithms”, Oxford University Press, 1st Edition,
2014.
2. Adam Drozdex, “Data Structures and algorithms in C++”, Cengage Learning, 4th Edition,
2013.
3. T.H. Cormen, C.E.Leiserson, R.L. Rivest and C.Stein, "Introduction to Algorithms",
Prentice Hall of India, 3rd Edition, 2012.
4. Mark Allen Weiss, “Data Structures and Algorithms in C++”, Pearson Education,
3rd Edition, 2009.
5. E. Horowitz, S. Sahni and S. Rajasekaran, “Fundamentals of Computer Algorithms”,
University Press, 2nd Edition, 2008.
6. Alfred V. Aho, John E. Hopcroft, Jeffrey D. Ullman, “Data Structures and Algorithms”,
Pearson Education, Reprint 2006.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

UNIT I ROLE OF ALGORITHMS IN COMPUTING & COMPLEXITY 9


ANALYSIS
Algorithms – Algorithms as a Technology -Time and Space complexity of algorithms- Asymptotic analysis-
Average and worst-case analysis-Asymptotic notation-Importance of efficient algorithms-Program performance
measurement - Recurrences: The Substitution Method – The Recursion- Tree Method- Data structures and
algorithms.

1.1. Algorithms – Algorithms as a Technology

Suppose computers were infinitely fast and computer memory was free. Would you have any reason to
study algorithms? The answer is yes, if for no other reason than that you would still like to demonstrate
that your solution method terminates and does so with the correct answer.

If computers were infinitely fast, any correct method for solving a problem would do. You would
probably want your implementation to be within the bounds of good software engineering practice (i.e.,
well designed and documented), but you would most often use whichever method was the easiest to
implement. Of course, computers may be fast, but they are not infinitely fast. And memory may be cheap,
but it is not free. Computing time is therefore a bounded resource, and so is space in memory. These
resources should be used wisely, and algorithms that are efficient in terms of time or space will help you
do so.

Efficiency: Algorithms devised to solve the same problem often differ dramatically in their efficiency.
These differences can be much more significant than differences due to hardware and software.

As an example, in Chapter 2, we will see two algorithms for sorting. The first, known as insertion
sort, takes time roughly equal to c1n2 to sort n items, where c1 is a constant that does not depend on n. That
is, it takes time roughly proportional to n2. The second, merge sort, takes time roughly equal to c2n lg n,
where lg n stands for log2 n and c2 is another constant that also does not depend on n. Insertion sort
usually has a smaller constant factor than merge sort, so that c1 < c2. We shall see that the constant factors
can be far less significant in the running time than the dependence on the input size n. Where merge sort
has a factor of lg n in its running time, insertion sort has a factor of n, which is much larger. Although
insertion sort is usually faster than merge sort for small input sizes, once the input size n becomes large
enough, merge sort's advantage of lg n vs. n will more than compensate for the difference in constant
factors. No matter how much smaller c1 is than c2, there will always be a crossover point beyond which
merge sort is faster.

For a concrete example, let us pit a faster computer (computer A) running insertion sort against a
slower computer (computer B) running merge sort. They each must sort an array of one million numbers.
Suppose that computer A executes one billion instructions per second and computer B executes only ten
million instructions per second, so that computer A is 100 times faster than computer B in raw computing

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

power. To make the difference even more dramatic, suppose that the world's craftiest programmer codes
insertion sort in machine language for computer A, and the resulting code requires 2n2 instructions to sort
n numbers. (Here, c1 = 2.) Merge sort, on the other hand, is programmed for computer B by an average
programmer using a high-level language with an inefficient compiler, with the resulting code taking 50n
lg n instructions (so that c2 = 50). To sort one million numbers, computer A takes

while computer B takes

By using an algorithm whose running time grows more slowly, even with a poor compiler, computer B
runs 20 times faster than computer A! The advantage of merge sort is even more pronounced when we
sort ten million numbers: where insertion sort takes approximately 2.3 days, merge sort takes under 20
minutes. In general, as the problem size increases, so does the relative advantage of merge sort.

Algorithms and other technologies

The example above shows that algorithms, like computer hardware, are a technology. Total system
performance depends on choosing efficient algorithms as much as on choosing fast hardware. Just as rapid
advances are being made in other computer technologies, they are being made in algorithms as well.

You might wonder whether algorithms are truly that important on contemporary computers in light of
other advanced technologies, such as

 hardware with high clock rates, pipelining, and superscalar architectures,


 easy-to-use, intuitive graphical user interfaces (GUIs),
 object-oriented systems, and
 local-area and wide-area networking.

The answer is yes. Although there are some applications that do not explicitly require algorithmic
content at the application level (e.g., some simple web-based applications), most also require a degree of
algorithmic content on their own. For example, consider a web-based service that determines how to travel
from one location to another. (Several such services existed at the time of this writing.) Its implementation
would rely on fast hardware, a graphical user interface, wide-area networking, and also possibly on object
orientation. However, it would also require algorithms for certain operations, such as finding routes
(probably using a shortest-path algorithm), rendering maps, and interpolating addresses.

Moreover, even an application that does not require algorithmic content at the application level relies

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

heavily upon algorithms. Does the application rely on fast hardware? The hardware design used
algorithms. Does the application rely on graphical user interfaces? The design of any GUI relies on
algorithms. Does the application rely on networking? Routing in networks relies heavily on algorithms.
Was the application written in a language other than machine code? Then it was processed by a compiler,
interpreter, or assembler, all of which make extensive use of algorithms. Algorithms are at the core of
most technologies used in contemporary computers.

Furthermore, with the ever-increasing capacities of computers, we use them to solve larger problems
than ever before. As we saw in the above comparison between insertion sort and merge sort, it is at larger
problem sizes that the differences in efficiencies between algorithms become particularly prominent.

1.2. Time and Space complexity of algorithms

Generally, there is always more than one way to solve a problem in computer science with different
algorithms. Therefore, it is highly required to use a method to compare the solutions in order to judge
which one is more optimal. The method must be:

 Independent of the machine and its configuration, on which the algorithm is running on.
 Shows a direct correlation with the number of inputs.
 Can distinguish two algorithms clearly without ambiguity.

Time Complexity: The time complexity of an algorithm quantifies the amount of time taken by an
algorithm to run as a function of the length of the input. Note that the time to run is a function of the
length of the input and not the actual execution time of the machine on which the algorithm is running on.

In order to calculate time complexity on an algorithm, it is assumed that a constant time c is taken to
execute one operation, and then the total operations for an input length on N are calculated. Consider an
example to understand the process of calculation: Suppose a problem is to find whether a pair (X, Y)
exists in an array, A of N elements whose sum is Z. The simplest idea is to consider every pair and check
if it satisfies the given condition or not.

The pseudo-code is as follows:

int a[n];
for(int i = 0;i < n;i++)
cin >> a[i]

for(int i = 0;i < n;i++)


for(int j = 0;j < n;j++)
if(i!=j && a[i]+a[j] == z)
return true

return false

Assuming that each of the operations in the computer takes approximately constant time, let it be c.
The number of lines of code executed actually depends on the value of Z. During analyses of the

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

algorithm, mostly the worst-case scenario is considered, i.e., when there is no pair of elements with sum
equals Z. In the worst case,

 N*c operations are required for input.


 The outer loop i loop runs N times.
 For each i, the inner loop j loop runs N times.

So total execution time is N*c + N*N*c + c. Now ignore the lower order terms since the lower order
terms are relatively insignificant for large input, therefore only the highest order term is taken (without
constant) which is N*N in this case. Different notations are used to describe the limiting behavior of a
function, but since the worst case is taken so big-O notation will be used to represent the time complexity.

Hence, the time complexity is O(N2) for the above algorithm. Note that the time complexity is solely
based on the number of elements in array A i.e the input length, so if the length of the array will increase
the time of execution will also increase.

Order of growth is how the time of execution depends on the length of the input. In the above example, it
is clearly evident that the time of execution quadratically depends on the length of the array. Order of
growth will help to compute the running time with ease.

Another Example: Let’s calculate the time complexity of the below algorithm:

count = 0
for (int i = N; i > 0; i /= 2)
for (int j = 0; j < i; j++)
count++;

This is a tricky case. In the first look, it seems like the complexity is O(N * log N). N for the j′s loop
and log(N) for i′s loop. But it’s wrong. Let’s see why.

Think about how many times count++ will run.

 When i = N, it will run N times.


 When i = N / 2, it will run N / 2 times.
 When i = N / 4, it will run N / 4 times.
 And so on.

The total number of times count++ will run is N + N/2 + N/4+…+1= 2 * N. So the time complexity
will be O(N).

Some general time complexities are listed below with the input range for which they are accepted in
competitive programming:

Input Worst Accepted Time


Usually type of solutions
Length Complexity

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Recursion and backtracking


10 -12 O(N!)

15-18 O(2N * N) Recursion, backtracking, and bit manipulation


18-22 O(2N * N) Recursion, backtracking, and bit manipulation
30-40 O(2N/2 * N) Meet in the middle, Divide and Conquer
100 O(N4) Dynamic programming, Constructive
400 O(N3) Dynamic programming, Constructive
Dynamic programming, Binary Search,
2K O(N2* log N) Sorting,
Divide and Conquer
Dynamic programming, Graph, Trees,
Constructive
10K O(N2)

1M O(N* log N) Sorting, Binary Search, Divide and Conquer


100M O(N), O(log N), O(1) Constructive, Mathematical, Greedy Algorithms

Space Complexity: The space complexity of an algorithm quantifies the amount of space taken by an
algorithm to run as a function of the length of the input. Consider an example: Suppose a problem to find
the frequency of array elements.

The pseudo-code is as follows:

int freq[n];
int a[n];

for(int i = 0; i<n; i++)


{
cin>>a[i];
freq[a[i]]++;
}

Here two arrays of length N, and variable i are used in the algorithm so, the total space used is N * c +
N * c + 1 * c = 2N * c + c, where c is a unit space taken. For many inputs, constant c is insignificant, and
it can be said that the space complexity is O(N).

There is also auxiliary space, which is different from space complexity. The main difference is
where space complexity quantifies the total space used by the algorithm, auxiliary space quantifies the
extra space that is used in the algorithm apart from the given input. In the above example, the auxiliary
space is the space used by the freq[] array because that is not part of the given input. So total auxiliary
space is N * c + c which is O(N) only.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

1.3. Asymptotic analysis-Average and worst-case analysis

Worst Case Analysis (Usually Done) : In the worst-case analysis, we calculate the upper bound on the
running time of an algorithm. We must know the case that causes a maximum number of operations to be
executed. For Linear Search, the worst case happens when the element to be searched (x in the above
code) is not present in the array. When x is not present, the search() function compares it with all the
elements of arr[] one by one. Therefore, the worst-case time complexity of linear search would be Θ(n).

Average Case Analysis (Sometimes done) : In average case analysis, we take all possible inputs and
calculate computing time for all of the inputs. Sum all the calculated values and divide the sum by the
total number of inputs. We must know (or predict) the distribution of cases. For the linear search problem,
let us assume that all cases are uniformly distributed (including the case of x not being present in the
array). So we sum all the cases and divide the sum by (n+1). Following is the value of average-case time
complexity.

Average Case Time =

= Θ(n)

Best Case Analysis (Bogus) : In the best case analysis, we calculate the lower bound on the running time
of an algorithm. We must know the case that causes a minimum number of operations to be executed. In
the linear search problem, the best case occurs when x is present at the first location. The number of
operations in the best case is constant (not dependent on n). So time complexity in the best case would be
Θ(1) Most of the times, we do worst-case analysis to analyze algorithms. In the worst analysis, we
guarantee an upper bound on the running time of an algorithm which is good information.
The average case analysis is not easy to do in most practical cases and it is rarely done. In the average case
analysis, we must know (or predict) the mathematical distribution of all possible inputs.
The Best Case analysis is bogus. Guaranteeing a lower bound on an algorithm doesn’t provide any
information as in the worst case, an algorithm may take years to run.
For some algorithms, all the cases are asymptotically the same, i.e., there are no worst and best cases. For
example, Merge Sort. Merge Sort does Θ(nLogn) operations in all cases. Most of the other sorting
algorithms have worst and best cases. For example, in the typical implementation of Quick Sort (where
pivot is chosen as a corner element), the worst occurs when the input array is already sorted and the best
occurs when the pivot elements always divide the array into two halves. For insertion sort, the worst case

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

occurs when the array is reverse sorted and the best case occurs when the array is sorted in the same order
as output.

1.4. Asymptotic notation

The main idea of asymptotic analysis is to have a measure of the efficiency of algorithms that don’t
depend on machine-specific constants and don’t require algorithms to be implemented and time taken by
programs to be compared. Asymptotic notations are mathematical tools to represent the time complexity
of algorithms for asymptotic analysis. The following 3 asymptotic notations are mostly used to represent
the time complexity of algorithms.

In case you wish to attend live classes with experts, please refer DSA Live Classes for Working
Professionals and Competitive Programming Live for Students.

1) Θ Notation: The theta notation bounds a function from above and below, so it defines exact asymptotic
behavior. A simple way to get the Theta notation of an expression is to drop low-order terms and ignore
leading constants.

For example, consider the following expression.


3n3 + 6n2 + 6000 = Θ(n3)

Dropping lower order terms is always fine because there will always be a number(n) after which Θ(n 3) has
higher values than Θ(n2) irrespective of the constants involved.

For a given function g(n), we denote Θ(g(n)) is following set of functions.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Θ(g(n)) = {f(n): there exist positive constants c1, c2 and n0 such


that 0 <= c1*g(n) <= f(n) <= c2*g(n) for all n >= n0}

The above definition means, if f(n) is theta of g(n), then the value f(n) is always between c1*g(n) and
c2*g(n) for large values of n (n >= n0). The definition of theta also requires that f(n) must be non-negative
for values of n greater than n0.

2) Big O Notation: The Big O notation defines an upper bound of an algorithm, it bounds a function only
from above. For example, consider the case of Insertion Sort. It takes linear time in the best case and
quadratic time in the worst case. We can safely say that the time complexity of Insertion sort is O(n^2).
Note that O(n^2) also covers linear time.

If we use Θ notation to represent time complexity of Insertion sort, we have to use two statements
for best and worst cases:
1. The worst-case time complexity of Insertion Sort is Θ(n^2).
2. The best case time complexity of Insertion Sort is Θ(n).

The Big O notation is useful when we only have an upper bound on the time complexity of an algorithm.
Many times we easily find an upper bound by simply looking at the algorithm.

O(g(n)) = { f(n): there exist positive constants c and


n0 such that 0 <= f(n) <= c*g(n) for
all n >= n0}

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

3) Ω Notation: Just as Big O notation provides an asymptotic upper bound on a function, Ω notation
provides an asymptotic lower bound. Ω Notation can be useful when we have a lower bound on the time
complexity of an algorithm. As discussed in the previous post, the best case performance of an algorithm
is generally not useful, the Omega notation is the least used notation among all three.

For a given function g(n), we denote by Ω(g(n)) the set of functions.

Ω (g(n)) = {f(n): there exist positive constants c and


n0 such that 0 <= c*g(n) <= f(n) for
all n >= n0}.

Let us consider the same Insertion sort example here. The time complexity of Insertion Sort can be written
as Ω(n), but it is not very useful information about insertion sort, as we are generally interested in worst-
case and sometimes in the average case.

Properties of Asymptotic Notations :


As we have gone through the definition of these three notations let’s now discuss some important
properties of those notations.

1. General Properties :

If f(n) is O(g(n)) then a*f(n) is also O(g(n)) ; where a is a constant.

Example: f(n) = 2n²+5 is O(n²)


then 7*f(n) = 7(2n²+5) = 14n²+35 is also O(n²) .

Similarly, this property satisfies both Θ and Ω notation.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

We can say
If f(n) is Θ(g(n)) then a*f(n) is also Θ(g(n)) ; where a is a constant.
If f(n) is Ω (g(n)) then a*f(n) is also Ω (g(n)) ; where a is a constant.

2. Transitive Properties :

If f(n) is O(g(n)) and g(n) is O(h(n)) then f(n) = O(h(n)) .

Example: if f(n) = n, g(n) = n² and h(n)=n³


n is O(n²) and n² is O(n³)
then n is O(n³)

Similarly, this property satisfies both Θ and Ω notation.

We can say
If f(n) is Θ(g(n)) and g(n) is Θ(h(n)) then f(n) = Θ(h(n)) .
If f(n) is Ω (g(n)) and g(n) is Ω (h(n)) then f(n) = Ω (h(n))

3. Reflexive Properties :

Reflexive properties are always easy to understand after transitive.

If f(n) is given then f(n) is O(f(n)). Since MAXIMUM VALUE OF f(n) will be f(n) ITSELF !

Hence x = f(n) and y = O(f(n) tie themselves in reflexive relation always.

Example: f(n) = n² ; O(n²) i.e O(f(n))

Similarly, this property satisfies both Θ and Ω notation.

We can say that:

If f(n) is given then f(n) is Θ(f(n)).

If f(n) is given then f(n) is Ω (f(n)).

4. Symmetric Properties :

If f(n) is Θ(g(n)) then g(n) is Θ(f(n)) .

Example: f(n) = n² and g(n) = n²


then f(n) = Θ(n²) and g(n) = Θ(n²)

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

This property only satisfies for Θ notation.

5. Transpose Symmetric Properties :

If f(n) is O(g(n)) then g(n) is Ω (f(n)).

Example: f(n) = n , g(n) = n²


then n is O(n²) and n² is Ω (n)

This property only satisfies O and Ω notations.

1.5. Importance of efficient algorithms

In computer science, algorithmic efficiency is a characteristic of an algorithm that is related to the


number of computational resources required by the algorithm. An algorithm’s resource use must be
evaluated, and the efficiency of an algorithm may be assessed based on the use of various resources. For a
recurring or continuous process, algorithmic efficiency is comparable to engineering performance. We
want to use as few resources as possible to maximize efficiency. However, because various resources,
such as time and space complexity, cannot be directly compared, which of the proposed algorithm is
judged to be more efficient typically relies on whatever efficiency metric is considered more significant.

Efficiency Factors

Efficiency is dependent on two factors:

Space Efficiency

In some cases, the amount of space/memory consumed has to be examined. For example, in dealing
with huge amounts of data or in programming embedded systems, memory consumed must be analyzed.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Space/memory usage components are:

 Space of Instruction: Compiler, compiler settings, and target machine (CPU), all have an impact on space

of instruction.

 Data Space: Data size/dynamically allocated memory, static program variables are all factors affecting
data space.
 Stack Space at Runtime: Compiler, run-time function calls and recursion, local variables, and
arguments all have an impact on stack space at runtime.

Time Efficiency

Obviously, the faster a program/function completes its objective, the better the algorithm. The actual
running time is determined by a number of factors:

 Computer’s Speed: processor (not simply clock speed), I/O, and so on.
 Compiler: The compiler, as well as the compiler parameters.
 Amount of Data: The amount of data, for example, whether to search a lengthy or a short list.
 Actual Data: The actual data, such as whether the name is first or last in a sequential search.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Approaches to Time Efficiency

There are two ways to assess time complexity:

Asymptotic Categorization: This employs basic categories to offer a basic notion of performance, which is
also called an order of magnitude. If the algorithms are similar, the data amount is little, or performance is
essential, then the next method can be explored further.

Estimation of running time depends on two things, i.e., Code Analysis and Code Execution, which
are elaborated below:

Code Analysis: We can accomplish the following things by analyzing the code:

 Operation Counts: Choose the most frequently performed operation(s) and count how many times
each one is performed.
 Step Counts: Calculate the total number of steps and potentially lines of code that program executes.

Code Execution: We may accomplish the following by executing the code:

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

 Benchmarking: Running the software on a variety of data sets and comparing the results.
 Description: A report on the number of hours spent in each routine of a program, which is used to
identify and eliminate the program’s hot spots. This perception is frequently questioned. Although other
description modes give data in units other than time (for example, call counts) and/or at levels of
granularity besides per-routine, the concept is the same.

Three Cases of Time Efficiency

 Worst case
 Average case
 Best case

The worst-case scenario is evaluated since it provides an upper limit on projected performance.
Furthermore, the average scenario is typically more difficult to establish (it is more data dependent), and it
is frequently the same as the worst case. It may be essential to examine all three situations on some
occasions.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

1.6. Program performance measurement

Performance measurement and program evaluation can both help identify areas of programs that need
improvement and determine whether the program is achieving its goals or objectives. They serve different
but complimentary functions:

 Performance measurement is an ongoing process that monitors and reports on a program's progress
and accomplishments by using pre-selected performance measures.
 Program evaluation, however, uses measurement and analysis to answer specific questions about
how well a program is achieving its outcomes and why.

What is Program Evaluation?

Program evaluations are individual systematic studies conducted to assess how well a program is working
and why. EPA has used program evaluation to:

 Support new and innovative approaches and emerging practices


 Identify opportunities to improve efficiency and effectiness
 Continuously improve existing programs
 Subsequently, improve human health and the environment

What Types of Program Evaluations are there?

Program Evaluation:-

Program evaluations can assess the performance of a program at all stages of a program's
development. The type of program evaluation conducted aligns with the program's maturity (e.g.,
developmental, implementation, or completion) and is driven by the purpose for conducting the evaluation
and the questions that it seeks to answer. The purpose of the program evaluation determines which type of
evaluation is needed.

Design Evaluation:-

A design evaluation is conducted early in the planning stages or implementation of a program. It


helps to define the scope of a program or project and to identify appropriate goals and objectives. Design
evaluations can also be used to pre-test ideas and strategies.

Process Evaluation:-

A process evaluation assesses whether a program or process is implemented as designed or


operating as intended and identifies opportunities for improvement. Process evaluations often begin with
an analysis of how a program currently operates. Process evaluations may also assess whether program

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

activities and outputs conform to statutory and regulatory requirements, EPA policies, program design or
customer expectations.

Outcome Evaluations:-

Outcome evaluations examine the results of a program (intended or unintended) to determine the
reasons why there are differences between the outcomes and the program's stated goals and objectives
(e.g., why the number and quality of permits issued exceeded or fell short of the established goal?).
Outcome evaluations sometimes examine program processes and activities to better understand how
outcomes are achieved and how quality and productivity could be improved.

Impact Evaluation:-

An impact evaluation is a subset of an outcome evaluation. It assesses the causal links between
program activities and outcomes. This is achieved by comparing the observed outcomes with an estimate
of what would have happened if the program had not existed (e.g., would the water be swimmable if the
program had not been instituted).

Cost-Effectiveness Evaluation:-

Cost-effectiveness evaluations identify program benefits, outputs or outcomes and compare them
with the internal and external costs of the program.

What is performance measurement?

Performance measurement is a way to continuously monitor and report a program's progress and
accomplishments, using pre-selected performance measures. By establishing program measures, offices
can gauge whether their program is meeting their goals and objectives. Performance measures help
programs understand "what" level of performance is achieved.

How do we determine good measures?

Measurement is essential to making cost-effective decisions. We strive to meet three key criteria in our
measurement work:

 Is it meaningful?
o Measurement should be consistent and comparable to help sustain learning.
 Is it credible?
o Effective measurement should withstand reasonable scrutiny.
 Is it practical?
o Measurement should be scaled to an agency's needs and budgetary constraints.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

How is performance measurement different from program evaluation?

 A program sets performance measures as a series of goals to meet over time. Measurement data
can be used to identify/flag areas of increasing or decreasing performance that may warrant further
investigation or evaluation. Program evaluations assess whether the program is meeting those
performance measures but also look at why they are or are not meeting them.
 For example, imagine you bought a new car that is supposed to get 30 miles per gallon. But say,
you notice that you are only getting 20 miles per gallon. That's a performance measurement. You
looked at whether your car was performing where it should be. So what do you do next? You
would take it to a mechanic. The mechanic's analysis and recommendations would be the program
evaluation because the mechanic would diagnose why the car is not performing as well as it
should.

1.7. Recurrences: The Substitution Method

1) Substitution Method: We make a guess for the solution and then we use mathematical induction to
prove the guess is correct or incorrect.

For example consider the recurrence T(n) = 2T(n/2) + n

We guess the solution as T(n) = O(nLogn). Now we use induction


to prove our guess.

We need to prove that T(n) <= cnLogn. We can assume that it is true
for values smaller than n.

T(n) = 2T(n/2) + n
<= 2cn/2Log(n/2) + n
= cnLogn - cnLog2 + n
= cnLogn - cn + n
<= cnLogn

Master Method: Master Method is a direct way to get the solution. The master method works only for
following type of recurrences or for recurrences that can be transformed to following type.

T(n) = aT(n/b) + f(n) where a >= 1 and b > 1

There are following three cases:


1. If f(n) = O(nc) where c < Logba then T(n) = Θ(nLogba)

2. If f(n) = Θ(nc) where c = Logba then T(n) = Θ(ncLog n)

How does this work?

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Master method is mainly derived from recurrence tree method. If we draw recurrence tree of T(n) =
aT(n/b) + f(n), we can see that the work done at root is f(n) and work done at all leaves is Θ(n c) where c is
Logba. And the height of recurrence tree is Logbn

In recurrence tree method, we calculate total work done. If the work done at leaves is polynomially more,
then leaves are the dominant part, and our result becomes the work done at leaves (Case 1). If work done
at leaves and root is asymptotically same, then our result becomes height multiplied by work done at any
level (Case 2). If work done at root is asymptotically more, then our result becomes work done at root
(Case 3).

Examples of some standard algorithms whose time complexity can be evaluated using Master
Method
Merge Sort: T(n) = 2T(n/2) + Θ(n). It falls in case 2 as c is 1 and Log ba] is also 1. So the solution is
Θ(n Logn)

Binary Search: T(n) = T(n/2) + Θ(1). It also falls in case 2 as c is 0 and Logba is also 0. So the
solution is Θ(Logn)

Notes:
1) It is not necessary that a recurrence of the form T(n) = aT(n/b) + f(n) can be solved using Master
Theorem. The given three cases have some gaps between them. For example, the recurrence T(n) =
2T(n/2) + n/Logn cannot be solved using master method.

2) Case 2 can be extended for f(n) = Θ(ncLogkn)


If f(n) = Θ(ncLogkn) for some constant k >= 0 and c = Logba, then T(n) = Θ(ncLogk+1n)

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

1.8. The Recursion- Tree Method

Recursion Tree Method is a pictorial representation of an iteration method which is in the form of a
tree where at each level nodes are expanded.

2. In general, we consider the second term in recurrence as root.

3. It is useful when the divide & Conquer algorithm is used.

4. It is sometimes difficult to come up with a good guess. In Recursion tree, each root and child represents
the cost of a single subproblem.

5. We sum the costs within each of the levels of the tree to obtain a set of pre-level costs and then sum all
pre-level costs to determine the total cost of all levels of the recursion.

6. A Recursion Tree is best used to generate a good guess, which can be verified by the Substitution
Method.

Example 1

Consider T (n) = 2T + n2

We have to obtain the asymptotic bound using recursion tree method.

Solution: The Recursion tree for the above recurrence is

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Recurrence Tree Method: In this method, we draw a recurrence tree and calculate the time taken by
every level of tree. Finally, we sum the work done at all levels. To draw the recurrence tree, we start from
the given recurrence and keep drawing till we find a pattern among levels. The pattern is typically a
arithmetic or geometric series.

For example consider the recurrence relation


T(n) = T(n/4) + T(n/2) + cn2

cn2
/ \
T(n/4) T(n/2)

If we further break down the expression T(n/4) and T(n/2),


we get following recursion tree.

cn2
/ \
c(n2)/16 c(n2)/4
/ \ / \
T(n/16) T(n/8) T(n/8) T(n/4)
Breaking down further gives us following
cn2
/ \
c(n2)/16 c(n2)/4
/ \ / \
c(n )/256 c(n )/64 c(n2)/64 c(n2)/16
2 2

/ \ / \ / \ / \

To know the value of T(n), we need to calculate sum of tree


nodes level by level. If we sum the above tree level by level,
we get the following series
T(n) = c(n^2 + 5(n^2)/16 + 25(n^2)/256) + ....
The above series is geometrical progression with ratio 5/16.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

To get an upper bound, we can sum the infinite series.


We get the sum as (n2)/(1 - 5/16) which is O(n2)

1.9. Data structures and algorithms

Data Structures are the programmatic way of storing data so that data can be used
efficiently. Almost every enterprise application uses various types of data structures in one or
the other way. This tutorial will give you a great understanding on Data Structures needed to
understand the complexity of enterprise level applications and need of algorithms, and data
structures.

Why to Learn Data Structure and Algorithms?

As applications are getting complex and data rich, there are three common problems that
applications face now-a-days.

 Data Search − Consider an inventory of 1 million(106) items of a store. If the


application is to search an item, it has to search an item in 1 million(10 6) items every
time slowing down the search. As data grows, search will become slower.
 Processor speed − Processor speed although being very high, falls limited if the data
grows to billion records.
 Multiple requests − As thousands of users can search data simultaneously on a web
server, even the fast server fails while searching the data.

To solve the above-mentioned problems, data structures come to rescue. Data can be
organized in a data structure in such a way that all items may not be required to be searched,
and the required data can be searched almost instantly.

Applications of Data Structure and Algorithms

Algorithm is a step-by-step procedure, which defines a set of instructions to be executed in a


certain order to get the desired output. Algorithms are generally created independent of
underlying languages, i.e. an algorithm can be implemented in more than one programming
language.

From the data structure point of view, following are some important categories of algorithms −

 Search − Algorithm to search an item in a data structure.


 Sort − Algorithm to sort items in a certain order.
 Insert − Algorithm to insert item in a data structure.
 Update − Algorithm to update an existing item in a data structure.
 Delete − Algorithm to delete an existing item from a data structure.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

The following computer problems can be solved using Data Structures −

 Fibonacci number series


 Knapsack problem
 Tower of Hanoi
 All pair shortest path by Floyd-Warshall
 Shortest path by Dijkstra
 Project scheduling

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

UNIT II HIERARCHICAL DATA STRUCTURES 9


Binary Search Trees: Basics – Querying a Binary search tree – Insertion and Deletion- Red Black trees:
Properties of Red-Black Trees – Rotations – Insertion – Deletion -B-Trees: Definition of B - trees – Basic
operations on B-Trees – Deleting a key from a B-Tree- Heap – Heap Implementation – Disjoint Sets -
Fibonacci Heaps: structure – Mergeable-heap operations- Decreasing a key and deleting a node-Bounding
the maximum degree.
2.1. Binary Search Trees: Basics

Binary Search Tree is a node-based binary tree data structure which has the following
properties:

 The left subtree of a node contains only nodes with keys lesser than the node’s key.
 The right subtree of a node contains only nodes with keys greater than the node’s key.
 The left and right subtree each must also be a binary search tree.

2.2. Querying a Binary search tree – Insertion and Deletion

 The left subtree of a node contains only nodes with keys lesser than the node’s key.
 The right subtree of a node contains only nodes with keys greater than the node’s key.
 The left and right subtree each must also be a binary search tree.
There must be no duplicate nodes.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

The above properties of Binary Search Tree provides an ordering among keys so that the
operations like search, minimum and maximum can be done fast. If there is no ordering, then
we may have to compare every key to search for a given key.

Searching a key : For searching a value, if we had a sorted array we could have performed a
binary search. Let’s say we want to search a number in the array what we do in binary search is
we first define the complete list as our search space, the number can exist only within the search
space. Now we compare the number to be searched or the element to be searched with the mid
element of the search space or the median and if the record being searched is lesser we go
searching in the left half else we go searching in the right half, in case of equality we have
found the element. In binary search we start with ‘n’ elements in search space and then if the
mid element is not the element that we are looking for, we reduce the search space to ‘n/2’ and
we go on reducing the search space till we either find the record that we are looking for or we
get to only one element in search space and be done with this whole reduction.

Search operation in binary search tree will be very similar. Let’s say we want to search
for the number, what we’ll do is we’ll start at the root, and then we will compare the value to be

Searched with the value of the root if it’s equal we are done with the search if it’s lesser
we know that we need to go to the left subtree because in a binary search tree all the elements in
the left subtree are lesser and all the elements in the right subtree are greater. Searching an
element in the binary search tree is basically this traversal in which at each step we will go
either towards left or right and hence in at each step we discard one of the sub-trees. If the tree
is balanced, we call a tree balanced if for all nodes the difference between the heights of left and
right subtrees is not greater than one, we will start with a search space of ‘n’nodes and when we
will discard one of the sub-trees we will discard ‘n/2’ nodes so our search space will be reduced
to ‘n/2’ and then in the next step we will reduce the search space to ‘n/4’ and we will go on

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

reducing like this till we find the element or till our search space is reduced to only one node.
The search here is also a binary search and that’s why the name binary search tree.

Illustration to search 6 in below tree:


1. Start from the root.
2. Compare the searching element with root, if less than root, then recurse for left, else recurse
for right.
3. If the element to search is found anywhere, return true, else return false.

Insertion of a key :-
A new key is always inserted at the leaf. We start searching a key from the root until we hit a
leaf node. Once a leaf node is found, the new node is added as a child of the leaf node.

100 100
/ \ Insert 40 / \
20 500 ---------> 20 500
/ \ / \
10 30 10 30
\
40

Illustration to insert 2 in below tree:


1. Start from the root.
2. Compare the inserting element with root, if less than root, then recurse for left, else recurse
for right.
3. After reaching the end, just insert that node at left(if less than current) else right.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Time Complexity: The worst-case time complexity of search and insert operations is O(h)
where h is the height of the Binary Search Tree. In the worst case, we may have to travel from
root to the deepest leaf node. The height of a skewed tree may become n and the time
complexity of search and insert operation may become O(n).

Node to be deleted is the leaf: Simply remove from the tree.

50 50
/ \ delete(20) / \
30 70 ---------> 30 70
/ \ / \ \ / \
20 40 60 80 40 60 80

2) Node to be deleted has only one child: Copy the child to the node and delete the child

50 50
/ \ delete(30) / \
30 70 ---------> 40 70
\ / \ / \
40 60 80 60 80

3) Node to be deleted has two children: Find inorder successor of the node. Copy contents of
the inorder successor to the node and delete the inorder successor. Note that inorder predecessor
can also be used.

50 60
/ \ delete(50) / \
40 70 ---------> 40 70
/ \ \
60 80 80

The important thing to note is, inorder successor is needed only when the right child is not
empty. In this particular case, inorder successor can be obtained by finding the minimum value
in the right child of the node.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Time Complexity: The worst case time complexity of delete operation is O(h) where h is the
height of the Binary Search Tree. In worst case, we may have to travel from the root to the
deepest leaf node. The height of a skewed tree may become n and the time complexity of delete
operation may become O(n)

2.3. Red Black trees: Properties of Red-Black Trees – Rotations – Insertion – Deletion

A red-black tree is a kind of self-balancing binary search tree where each node has an
extra bit, and that bit is often interpreted as the colour (red or black). These colours are used to
ensure that the tree remains balanced during insertions and deletions. Although the balance of
the tree is not perfect, it is good enough to reduce the searching time and maintain it around
O(log n) time, where n is the total number of elements in the tree. This tree was invented in
1972 by Rudolf Bayer.

It must be noted that as each node requires only 1 bit of space to store the colour
information, these types of trees show identical memory footprint to the classic (uncoloured)
binary search tree.

Rules That Every Red-Black Tree Follows:

1. Every node has a colour either red or black.


2. The root of the tree is always black.
3. There are no two adjacent red nodes (A red node cannot have a red parent or red child).
4. Every path from a node (including root) to any of its descendants NULL nodes has the
same number of black nodes.

Why Red-Black Trees?

Most of the BST operations (e.g., search, max, min, insert, delete.. etc) take O(h) time
where h is the height of the BST. The cost of these operations may become O(n) for a skewed
Binary tree. If we make sure that the height of the tree remains O(log n) after every insertion
and deletion, then we can guarantee an upper bound of O(log n) for all these operations. The
height of a Red-Black tree is always O(log n) where n is the number of nodes in the tree.

The AVL trees are more balanced compared to Red-Black Trees, but they may cause
more rotations during insertion and deletion. So if your application involves frequent insertions
and deletions, then Red-Black trees should be preferred. And if the insertions and deletions are
less frequent and search is a more frequent operation, then AVL tree should be preferred over
Red-Black Tree.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

How does a Red-Black Tree ensure balance?

A simple example to understand balancing is, a chain of 3 nodes is not possible in the
Red-Black tree. We can try any combination of colours and see all of them violate Red-Black
tree property.

A chain of 3 nodes is not possible in Red-Black Trees.


Following are NOT Red-Black Trees
30 30 30
/ \ / \ / \
20 NIL 20 NIL 20 NIL
/ \ / \ / \
10 NIL 10 NIL 10 NIL
Violates Violates Violates
Property 4. Property 4 Property 3

Following are different possible Red-Black Trees with above 3 keys


20 20
/ \ / \
10 30 10 30
/ \ / \ / \ / \
NIL NIL NIL NIL NIL NIL NIL NIL

Interesting points about Red-Black Tree:

1. Black height of the red-black tree is the number of black nodes on a path from the root
node to a leaf node. Leaf nodes are also counted as black nodes. So, a red-black tree of
height h has black height >= h/2.
2. Height of a red-black tree with n nodes is h<= 2 log2(n + 1).
3. All leaves (NIL) are black.
4. The black depth of a node is defined as the number of black nodes from the root to that
node i.e the number of black ancestors.
5. Every red-black tree is a special case of a binary tree.

Black Height of a Red-Black Tree :

Black height is the number of black nodes on a path from the root to a leaf. Leaf nodes
are also counted black nodes. From the above properties 3 and 4, we can derive, a Red-Black
Tree of height h has black-height >= h/2.

Number of nodes from a node to its farthest descendant leaf is no more than twice as the
number of nodes to the nearest descendant leaf.

Every Red Black Tree with n nodes has height <= 2Log2(n+1)
This can be proved using the following facts:

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

1. For a general Binary Tree, let k be the minimum number of nodes on all root to NULL
paths, then n >= 2k – 1 (Ex. If k is 3, then n is at least 7). This expression can also be
written as k <= Log2(n+1).
2. From property 4 of Red-Black trees and above claim, we can say in a Red-Black Tree
with n nodes, there is a root to leaf path with at-most Log2(n+1) black nodes.
3. From property 3 of Red-Black trees, we can claim that the number of black nodes in a
Red-Black tree is at least ⌊ n/2 ⌋ where n is the total number of nodes.

From the above points, we can conclude the fact that Red Black Tree with n nodes has height
<= 2Log2(n+1).

Search Operation in Red-black Tree:

As every red-black tree is a special case of a binary tree so the searching algorithm of a red-
black tree is similar to that of a binary tree.

Algorithm:

searchElement (tree, val)


Step 1:
If tree -> data = val OR tree = NULL
Return tree
Else
If val < data
Return searchElement (tree -> left, val)
Else
Return searchElement (tree -> right, val)
[ End of if ]
[ End of if ]

Step 2: END

Example: Searching 11 in the following red-black tree.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Solution:

1. Start from the root.


2. Compare the inserting element with root, if less than root, then recurse for left, else
recurse for right.
3. If the element to search is found anywhere, return true, else return false.

Just follow the blue bubble.

In this post, we introduced Red-Black trees and discussed how balance is ensured. The hard part
is to maintain balance when keys are added and removed. We have also seen how to search an
element from the red-black tree. We will soon be discussing insertion and deletion operations in
coming posts on the Red-Black tree.

Rotations in Binary Search Tree

There are two types of rotations:

 Left Rotation
 Right Rotation

In left rotation, we assume that the right child is not null. Similarly, in the right rotation, we
assume that the left child is not null.

Consider the following tree:

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

After applying left rotation on the node x, the node y will become the new root of the
subtree and its left child will be x. And the previous left child of y will now become the right
child of x.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Now applying right rotation on the node y of the rotated tree, it will transform back to
the original tree.

So right rotation on the node y will make x the root of the tree, y will become x's right
child. And the previous right child of x will now become the left child of y.

Take a note that rotation doesn't violate the property of binary search trees.

The left grandchild of x (left child of the right child x) will become the right child of it after
rotation. We will do this but before doing this, let's mark the right child of x as y.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

LEFT_ROTATION(T, x)
y = x.right
x.right = y.left

The left child of y is going to be the right child of x - x.right = y.left. We also need
to change the parent of y.left to x. We will do this if the left child of y is not NULL.

if y.left != NULL
y.left.parent = x

Then we need to put y to the position of x. We will first change the parent of y to the
parent of x - y.parent = x.parent. After this, we will make the node x the child of y's parent
instead of y. We will do so by checking if y is the right or left child of its parent. We will also
check if y is the root of the tree.

y.parent = x.parent
if x.parent == NULL //x is root
T.root = y
elseif x == x.parent.left // x is left child
x.parent.left = y
else // x is right child
x.parent.right = y

At last, we need to make x the left child of y.

y.left = x
x.parent = y

LEFT_ROTATE(T, x)

y = x.right

x.right = y.left

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

if y.left != NULL

y.left.parent = x

y.parent = x.parent

if x.parent == NULL //x is root

T.root = y

elseif x == x.parent.left // x is left child

x.parent.left = y

else // x is right child

x.parent.right = y

y.left = x

x.parent = y

From the above code, you can easily see that rotation is a constant time taking process ( O(1)

).

INSERT(T, n)

y = T.NIL

temp = T.root

while temp != T.NIL

y = temp

if n.data < temp.data

temp = temp.left

else

temp = temp.right

n.parent = y

if y==T.NIL

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

T.root = n

else if n.data < y.data

y.left = n

else

y.right = n

n.left = T.NIL

n.right = T.NIL

n.color = RED

INSERT_FIXUP(T, n)

Here, we have used T.NIL instead of NULL unlike we do with normal binary search tree.
Also, those T.NIL are the leaves and they all are black, so there won't be a violation of property
3.

In the last few lines, we are making the left and right of the new node T.NIL and also
making it red. At last, we are calling the function to fix the violations of the red-black
properties. Rest of the code is similar to a normal binary search tree.

Let's focus on the function to fix the violations.

The property 4 will be violated when the parent of the inserted node is red. So, we will
fix the violations if the parent of the new node is red. At last, we will color the root black and
that will fix the violation of property 2 if it is violated. In the case of violation of property 4
(when the parent of the new node is red), the grandparent will be black.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

There can be six cases in the case of violation of the property 4. Let's look at the given
pictures first assuming that the parent of the node z is a left child of its parent which gives us the
first three cases. The other three cases will be symmetric when the node z will be the right child
of its parent.

The first case is when the uncle of z is also red. In this case, we will shift the red color
upward until there is no violation. Otherwise, if it reaches to the root, we can just color it black
without any consequences.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

So, the process is to make both the parent and the uncle of z black and its grandparent red. In
this way, the black height of any node won't be affected and we can successfully shift the red
color upward.

However, making the grandparent of z red might cause violation of the property 4 there. So, we
will do the fixup again on that node.

In the second case, the uncle of the node z is black and the node z is the right child.

In the third case, the uncle of the node z is black and the node z is the left child.

We can transform the second case into the third one by performing left rotation on the
parent of the node z. Since both z and its parent are red, so rotation won't affect the black height.

In case 3, we first color the parent of the node z black and its grandparent red and then
do a right rotation on the grandparent of the node z. This fixes the violation of properties
completely. This is shown in the picture given below.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Since we are making the parent of z black in both case 2 and case 3, the loop will stop in these
two cases.

Let's look at the following example first.

Similarly, there will be three cases when the parent of z will be the right child but those
cases will be symmetric to the above cases only with left and right exchanged.

2.4. B-Trees: Definition of B - trees – Basic operations on B-Trees – Deleting a key from a B-Tree

1. All leaves are at the same level.


2. A B-Tree is defined by the term minimum degree ‘t’. The value of t depends upon disk
block size.
3. Every node except root must contain at least t-1 keys. The root may contain minimum 1
key.
4. All nodes (including root) may contain at most 2*t – 1 keys.
5. Number of children of a node is equal to the number of keys in it plus 1.
6. All keys of a node are sorted in increasing order. The child between two keys k1 and k2
contains all keys in the range from k1 and k2.
7. B-Tree grows and shrinks from the root which is unlike Binary Search Tree. Binary
Search Trees grow downward and also shrink from downward.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

8. Like other balanced Binary Search Trees, time complexity to search, insert and delete is
O(log n).
9. Insertion of a Node in B-Tree happens only at Leaf Node.

Following is an example of B-Tree of minimum order 5. Note that in practical B-Trees, the
value of the minimum order is much more than 5.

We can see in the above diagram that all the leaf nodes are at the same level and all non-leaf
have no empty sub-tree and have keys one less than the number of their children.
Interesting Facts:

1. The minimum height of the B-Tree that can exist with n number of nodes and m is the
maximum number of children of a node.
2. The maximum height of the B-Tree that can exist with n number of nodes and d is the
minimum number of children that a non-root node.

Traversal in B-Tree: Traversal is also similar to Inorder traversal of Binary Tree. We start
from the leftmost child, recursively print the leftmost child, then repeat the same process for
remaining children and keys. In the end, recursively print the rightmost child.

Search Operation in B-Tree: Search is similar to the search in Binary Search Tree. Let the key
to be searched be k. We start from the root and recursively traverse down. For every visited
non-leaf node, if the node has the key, we simply return the node. Otherwise, we recur down to
the appropriate child (The child which is just before the first greater key) of the node. If we
reach a leaf node and don’t find k in the leaf node, we return NULL.

Logic: Searching a B-Tree is similar to searching a binary tree. The algorithm is similar and
goes with recursion. At each level, the search is optimized as if the key value is not present in
the range of parent then the key is present in another branch. As these values limit the search
they are also known as limiting value or separation value. If we reach a leaf node and don’t find
the desired key then it will display NULL.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Example: Searching 120 in the given B-Tree.

Solution:

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

In this example, we can see that our search was reduced by just limiting the chances
where the key containing the value could be present. Similarly if within the above example
we’ve to look for 180, then the control will stop at step 2 because the program will find that the
key 180 is present within the current node. And similarly, if it’s to seek out 90 then as 90 < 100
so it’ll go to the left subtree automatically and therefore the control flow will go similarly as
shown within the above example.

Insertion
1) Initialize x as root.
2) While x is not leaf, do following
..a) Find the child of x that is going to be traversed next. Let the child be y.
..b) If y is not full, change x to point to y.
..c) If y is full, split it and change x to point to one of the two parts of y. If k is smaller than mid
key in y, then set x as the first part of y. Else second part of y. When we split y, we move a key
from y to its parent x.
3) The loop in step 2 stops when x is leaf. x must have space for 1 extra key as we have been
splitting all nodes in advance. So simply insert k to x.

Let us understand the algorithm with an example tree of minimum degree ‘t’ as 3 and a
sequence of integers 10, 20, 30, 40, 50, 60, 70, 80 and 90 in an initially empty B-Tree.
Initially root is NULL. Let us first insert 10.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Let us now insert 20, 30, 40 and 50. They all will be inserted in root because the maximum
number of keys a node can accommodate is 2*t – 1 which is 5.

Let us now insert 60. Since root node is full, it will first split into two, then 60 will be inserted
into the appropriate child.

Let us now insert 70 and 80. These new keys will be inserted into the appropriate leaf without
any split.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Let us now insert 90. This insertion will cause a split. The middle key will go up to the parent.

Deletion from a B-tree is more complicated than insertion, because we can delete a key from
any node-not just a leaf—and when we delete a key from an internal node, we will have to
rearrange the node’s children.

As in insertion, we must make sure the deletion doesn’t violate the B-tree properties. Just as
we had to ensure that a node didn’t get too big due to insertion, we must ensure that a node
doesn’t get too small during deletion (except that the root is allowed to have fewer than the
minimum number t-1 of keys). Just as a simple insertion algorithm might have to back up if a
node on the path to where the key was to be inserted was full, a simple approach to deletion
might have to back up if a node (other than the root) along the path to where the key is to be
deleted has the minimum number of keys.

The deletion procedure deletes the key k from the subtree rooted at x. This procedure
guarantees that whenever it calls itself recursively on a node x, the number of keys in x is at
least the minimum degree t . Note that this condition requires one more key than the minimum
required by the usual B-tree conditions, so that sometimes a key may have to be moved into a
child node before recursion descends to that child. This strengthened condition allows us to
delete a key from the tree in one downward pass without having to “back up” (with one

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

exception, which we’ll explain). You should interpret the following specification for deletion
from a B-tree with the understanding that if the root node x ever becomes an internal node
having no keys (this situation can occur in cases 2c and 3b then we delete x, and x’s only child
x.c1 becomes the new root of the tree, decreasing the height of the tree by one and preserving
the property that the root of the tree contains at least one key (unless the tree is empty).

We sketch how deletion works with various cases of deleting keys from a B-tree.

1. If the key k is in node x and x is a leaf, delete the key k from x.

2. If the key k is in node x and x is an internal node, do the following.

a) If the child y that precedes k in node x has at least t keys, then find the predecessor k0 of k
in the sub-tree rooted at y. Recursively delete k0, and replace k by k0 in x. (We can find k0 and
delete it in a single downward pass.)

b) If y has fewer than t keys, then, symmetrically, examine the child z that follows k in node
x. If z has at least t keys, then find the successor k0 of k in the subtree rooted at z. Recursively
delete k0, and replace k by k0 in x. (We can find k0 and delete it in a single downward pass.)

c) Otherwise, if both y and z have only t-1 keys, merge k and all of z into y, so that x loses
both k and the pointer to z, and y now contains 2t-1 keys. Then free z and recursively delete k
from y.

3. If the key k is not present in internal node x, determine the root x.c(i) of the appropriate
subtree that must contain k, if k is in the tree at all. If x.c(i) has only t-1 keys, execute step 3a or
3b as necessary to guarantee that we descend to a node containing at least t keys. Then finish by
recursing on the appropriate child of x.

a) If x.c(i) has only t-1 keys but has an immediate sibling with at least t keys, give x.c(i) an
extra key by moving a key from x down into x.c(i), moving a key from x.c(i) ’s immediate left
or right sibling up into x, and moving the appropriate child pointer from the sibling into x.c(i).

b) If x.c(i) and both of x.c(i)’s immediate siblings have t-1 keys, merge x.c(i) with one
sibling, which involves moving a key from x down into the new merged node to become the
median key for that node.

Since most of the keys in a B-tree are in the leaves, deletion operations are most often used to
delete keys from leaves. The recursive delete procedure then acts in one downward pass through
the tree, without having to back up. When deleting a key in an internal node, however, the
procedure makes a downward pass through the tree but may have to return to the node from
which the key was deleted to replace the key with its predecessor or successor (cases 2a and
2b).

The following figures explain the deletion process.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

2.5. Heap – Heap Implementation

A Binary Heap is a Binary Tree with following properties.

1) It’s a complete tree (All levels are completely filled except possibly the last level and the last
level has all keys as left as possible). This property of Binary Heap makes them suitable to be
stored in an array.

2) A Binary Heap is either Min Heap or Max Heap. In a Min Binary Heap, the key at root must
be minimum among all keys present in Binary Heap. The same property must be recursively
true for all nodes in Binary Tree. Max Binary Heap is similar to MinHeap.

Examples of Min Heap:

10 10
/ \ / \
20 100 15 30
/ / \ / \
30 40 50 100 40

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

How is Binary Heap represented?


A Binary Heap is a Complete Binary Tree. A binary heap is typically represented as an array.

 The root element will be at Arr[0].


 Below table shows indexes of other nodes for the ith node, i.e., Arr[i]:

Arr[(i-1)/2] Returns the parent node


Arr[(2*i)+1] Returns the left child node
Arr[(2*i)+2] Returns the right child node

 The traversal method use to achieve Array representation is Level Order

Applications of Heaps:

1) Heap Sort: Heap Sort uses Binary Heap to sort an array in O(nLogn) time.

2) Priority Queue: Priority queues can be efficiently implemented using Binary Heap because
it supports insert(), delete() and extractmax(), decreaseKey() operations in O(logn) time.
Binomoial Heap and Fibonacci Heap are variations of Binary Heap. These variations perform
union also efficiently.

3) Graph Algorithms: The priority queues are especially used in Graph Algorithms like
Dijkstra’s Shortest Path and Prim’s Minimum Spanning Tree.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

4) Many problems can be efficiently solved using Heaps. See following for example.
a) K’th Largest Element in an array.
b) Sort an almost sorted array/
c) Merge K Sorted Arrays.

Operations on Min Heap:


1) getMini(): It returns the root element of Min Heap. Time Complexity of this operation
is O(1).

2) extractMin(): Removes the minimum element from MinHeap. Time Complexity of


this Operation is O(Logn) as this operation needs to maintain the heap property (by
calling heapify()) after removing root.

3) decreaseKey(): Decreases value of key. The time complexity of this operation is


O(Logn). If the decreases key value of a node is greater than the parent of the node, then
we don’t need to do anything. Otherwise, we need to traverse up to fix the violated heap
property.

4) insert(): Inserting a new key takes O(Logn) time. We add a new key at the end of the
tree. IF new key is greater than its parent, then we don’t need to do anything. Otherwise,
we need to traverse up to fix the violated heap property.

5) delete(): Deleting a key also takes O(Logn) time. We replace the key to be deleted
with minum infinite by calling decreaseKey(). After decreaseKey(), the minus infinite
value must reach root, so we call extractMin() to remove the key.

Below is the implementation of basic heap operations.

A max-heap is a complete binary tree in which the value in each internal node is greater
than or equal to the values in the children of that node. Mapping the elements of a heap into an
array is trivial: if a node is stored an index k, then its left child is stored at index 2k+1 and its
right child at index 2k+2.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

How is Max Heap represented?

A-Max Heap is a Complete Binary Tree. A-Max heap is typically represented as an


array. The root element will be at Arr[0]. Below table shows indexes of other nodes for the ith
node, i.e., Arr[i]:

Arr[(i-1)/2] Returns the parent node.


Arr[(2*i)+1] Returns the left child node.
Arr[(2*i)+2] Returns the right child node.

Operations on Max Heap are as follows:

 getMax(): It returns the root element of Max Heap. The Time Complexity of this
operation is O(1).
 extractMax(): Removes the maximum element from MaxHeap. The Time Complexity
of this Operation is O(Log n) as this operation needs to maintain the heap property by
calling the heapify() method after removing the root.
 insert(): Inserting a new key takes O(Log n) time. We add a new key at the end of the
tree. If the new key is smaller than its parent, then we don’t need to do anything.
Otherwise, we need to traverse up to fix the violated heap property.

2.6. Disjoint Sets

Disjoint sets are those sets whose intersection with each other results in a null set. In Set
theory, sometimes we notice that there are no common elements in two sets or we can say that
the intersection of the sets is an empty set or null set. This type of set is called a disjoint set.
For example, if we have X = {a, b, c} and Y = {d, e, f}, then we can say that the given two sets
are disjoint since there are no common elements in these two sets X and Y. In this article, you
will learn what disjoint set is, disjoint set union, Venn diagram, pairwise disjoint set, examples
in detail.

Two sets are said to be disjoint when they have no common element. If a collection has
two or more sets, the condition of disjointness will be the intersection of the entire collection
should be empty.

Yet, a group of sets may have a null intersection without being disjoint. Moreover, while
a group of fewer than two sets is trivially disjoint, since no pairs are there to compare, the
intersection of a group of one set is equal to that set, which may be non-empty. For example, the
three sets {11, 12}, {12, 13}, and {11, 13} have a null intersection but they are not disjoint.
There are no two disjoint sets available in this group. Also, the empty family of sets is pairwise
disjoint.

Consider an example, {1, 2, 3} and {4, 5, 6} are disjoint sets.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Two sets A and B are disjoint sets if the intersection of two sets is a null set or an empty set. In
other words, the intersection of a set is empty.

i.e. A ∩ B = ϕ

Properties of Intersection:

 Commutative: A ∩ B = B ∩ A
 Associative: A ∩ (B ∩ C) = (A ∩ B) ∩ C
 A∩∅=∅
 A∩B⊆A
 A∩A=A
 A ⊆ B if and only if A ∩ B = A

Disjoint Set Union

A disjoint set union is a binary operation on two sets. The elements of any disjoint union
can be described in terms of ordered pairs as (x, j), where j is the index that represents the origin
of the element x. With the help of this operation, we can join all the different (distinct) elements
of a pair of sets.

A disjoint union may indicate one of two conditions. Most commonly, it may intend the
union of two or more sets that are disjoint. Else if they are disjoint, then their disjoint union may
be produced by adjusting the sets to obtain them disjoint before forming the union of the altered
sets. For example, two sets may be presented as a disjoint set by exchanging each element by an
ordered pair of the element and a binary value symbolising whether it refers to the first or
second set. For groups of more than two sets, one may likewise substitute each element by an
ordered pair of the element and the list of the set that contains it.

The disjoint union is denoted as X U* Y = ( X x {0} ) U ( Y x {1} ) = X* U Y*

Assume that,

The disjoint union of sets X = ( a, b, c, d ) and Y = ( e, f, g, h ) is as follows:

X* = { (a, 0), (b, 0), (c, 0), (d, 0) } and Y* = { (e, 1), (f, 1), (g, 1), (h, 1) }

Then,

X U* Y = X* U Y*

Therefore, the disjoint union set is { (a, 0), (b, 0), (c, 0), (d, 0), (e, 1), (f, 1), (g, 1), (h, 1) }

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Pairwise Disjoint sets

We can proceed with the definition of a disjoint set to any group of sets. A collection of
sets is pairwise disjoint if any two sets in the collection are disjoint. It is also known as mutually
disjoint sets.

Let P be the set of any collection of sets and A and B.

i.e. A, B ∈ P. Then, P is known as pairwise disjoint if and only if A ≠ B. Therefore, A ∩ B = ϕ

Examples:

 P = { {1}, {2, 3}, {4, 5, 6} } is a pairwise disjoint set.


 P = { {1, 2}, {2, 3} } is not pairwise disjoint, since we have 2 as the common element in
two sets.

Are Two Null Sets Disjointed?

We know that two sets are disjoint if they don’t have any common elements in the set.
When we take the intersection of two empty sets, the resultant set is also an empty set. One can
easily prove that only the empty sets are disjoint from itself. The following theorem shows that
an empty set is disjoint with itself.

Theorem:

The empty set is disjoint with itself.

Ø⋂Ø=Ø

Difference between Joint and Disjoint Set

Consider two sets X and Y.

Assume that both the sets X and Y are non-empty sets. Thus, X ⋂ Y is also a non-empty
set, the sets are called joint set. In case, if X ⋂ Y results in an empty set, then it is called the
disjoint set.

For example: X = {1, 5, 7} and Y = {3, 5, 6}

X ∩ Y = {5}

Hence, X and Y are joint sets.

In case, if
X = {1, 5, 7} and Y = {2, 4, 6}

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

X∩Y=Ø
Therefore, X and Y are disjoint sets.

Disjoint Sets Examples

Question 1: Show that the given two sets are disjoint sets.

A = {5, 9, 12, 14}

B = {3, 6}

Solution:

Given: A = {5, 9, 12, 14}, B = {3, 6}

A ∩ B = {5, 9, 12, 14} ∩ {3, 6}

Here, set A and B do not have any common element

That is, A ∩ B = { }

The intersection of set A and set B gives an empty set.

Hence, the sets A and B are disjoint sets.

Question 2: Draw a disjoint set Venn diagram that represents the given two sets

X = {a, b, c, d, f} and Y = {e, g, h, i}

Solution:

Given: X = {a, b, c, d, f}, Y = {e, g, h, i}

In the given problem, we don’t have a common factor.

Therefore, the given sets are disjoint.

i.e. A ∩ B = { }

The disjoint set Venn diagram is represented as:

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

The Venn diagram clearly shows that the given sets are disjoint sets.

Keep visiting BYJU’S – The Learning App to learn more Maths-related concepts and problems.

2.7. Fibonacci Heaps: structure – Mergeable

A fibonacci heap is a data structure that consists of a collection of trees which follow
min heap or max heap property. We have already discussed min heap and max heap property
in the In a fibonacci heap, a node can have more than two children or no children at all. Also, it
has more efficient heap operations than that supported by the binomial and binary heaps.

The fibonacci heap is called a fibonacci heap because the trees are constructed in a way
such that a tree of order n has at least Fn+2 nodes in it, where Fn+2 is the (n + 2)th Fibonacci
number.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Fibonacci Heap

Properties of a Fibonacci Heap

Important properties of a Fibonacci heap are:

1. It is a set of min heap-ordered trees. (i.e. The parent is always smaller than the children.)
2. A pointer is maintained at the minimum element node.
3. It consists of a set of marked nodes. (Decrease key operation)
4. The trees within a Fibonacci heap are unordered but rooted.

Memory Representation of the Nodes in a Fibonacci Heap

The roots of all the trees are linked together for faster access. The child nodes of a parent node
are connected to each other through a circular doubly linked list as shown below.

There are two main advantages of using a circular doubly linked list.

1. Deleting a node from the tree takes O(1) time.


2. The concatenation of two such lists takes O(1) time.

Fibonacci Heap Structure

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Operations on a Fibonacci Heap


Insertion

Algorithm

insert(H, x)
degree[x] = 0
p[x] = NIL
child[x] = NIL
left[x] = x
right[x] = x
mark[x] = FALSE
concatenate the root list containing x with root list H
if min[H] == NIL or key[x] < key[min[H]]
then min[H] = x
n[H] = n[H] + 1

Inserting a node into an already existing heap follows the steps below.

1. Create a new node for the element.


2. Check if the heap is empty.
3. If the heap is empty, set the new node as a root node and mark it min.
4. Else, insert the node into the root list and update min.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Insertion Example

Find Min

The minimum element is always given by the min pointer.

Union

Union of two fibonacci heaps consists of following steps.

1. Concatenate the roots of both the heaps.


2. Update min by selecting a minimum key from the new root lists.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Union of two heaps

Extract Min

It is the most important operation on a fibonacci heap. In this operation, the node with minimum
value is removed from the heap and the tree is re-adjusted.

The following steps are followed:

1. Delete the min node.


2. Set the min-pointer to the next root in the root list.
3. Create an array of size equal to the maximum degree of the trees in the heap before deletion.
4. Do the following (steps 5-7) until there are no multiple roots with the same degree.
5. Map the degree of current root (min-pointer) to the degree in the array.
6. Map the degree of next root to the degree in array.
7. If there are more than two mappings for the same degree, then apply union operation to those
roots such that the min-heap property is maintained (i.e. the minimum is at the root).

An implementation of the above steps can be understood in the example below.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

1. We will perform an extract-min operation on the heap below.

Fibonacci Heap
2. Delete the min node, add all its child nodes to the root list and set the min-pointer to the
next root in the root list.

Delete the min node

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

3. The maximum degree in the tree is 3. Create an array of size 4 and map degree of the next roots
with the array.

4. Create an array
5. Here, 23 and 7 have the same degrees, so unite them.

6. Unite those having the same degrees

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

7. Again, 7 and 17 have the same degrees, so unite them as well.

8. Unite those having the same degrees

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

9. Map the next nodes.

10. Map the remaining nodes


11. Again, 52 and 21 have the same degree, so unite them

12. Unite those having the same degrees

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

13. Similarly, unite 21 and 18.

14. Unite those having the same degrees


15. Map the remaining root.

16. Map the remaining nodes

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

17. The final heap is.

18. Final fibonacci heap

2.8. Heap operations- Decreasing a key and deleting a node

Operations on Heaps

Heap is a very useful data structure that every programmer should know well. The heap data
structure is used in Heap Sort, Priority Queues. The understanding of heaps helps us to know
about memory management. In this blog, we will discuss the various operations of the heap data
structure. We have already discussed what are heaps, its structure, types, and its representation
in the array in the last blog. So let’s get started with the operations on a heap.

Operations on Heaps

The common operation involved using heaps are:

 Heapify → Process to rearrange the heap in order to maintain heap-property.


 Find-max (or Find-min) → find a maximum item of a max-heap, or a minimum item of a min-
heap, respectively.
 Insertion → Add a new item in the heap.
 Deletion → Delete an item from the heap.
 Extract Min-Max → Returning and deleting the maximum or minimum element in max-heap
and min-heap respectively.

Heapify

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

It is a process to rearrange the elements of the heap in order to maintain the heap property. It is
done when a certain node causes an imbalance in the heap due to some operation on that node.

The heapify can be done in two methodologies:

 up_heapify() → It follows the bottom-up approach. In this, we check if the nodes are following
heap property by going in the direction of rootNode and if nodes are not following the heap
property we do certain operations to let the tree follows the heap property.
 down_heapify() → It follows the top-down approach. In this, we check if the nodes are
following heap property by going in the direction of the leaf nodes and if nodes are not
following the heap property we do certain operations to let the tree follows the heap property.

Pseudo Code
void down_heapify(int heap[], int parent, int size)
{
largest = parent
leftChild = 2*parent + 1
rightChild = 2*parent + 2

if(leftChild < size && heap[leftChild] > heap[largest])


largest = leftChild
if(rightChild < size && heap[rightChild] > heap[largest])
largest = rightChild
if(parent != largest)
{
swap(heap[parent], heap[largest])
down_heapify(heap,largest,size)
}
}
void up_heapify(int heap[], int child)
{
parent = (child-1)/2;
if(heap[parent] < heap[child]) {
swap(heap[parent], heap[child])
up_heapify(heap,parent)
}
}

Insertion

The insertion in the heap follows the following steps

 Insert the new element at the end of the heap.


 Since the newly inserted element can distort the properties of the Heap. So, we need to perform
up_heapify() operation, in order to keep the properties of the heap in a bottom-up approach.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Initially the heap is as (It follows max-heap property)

9
/ \
5 3
/\
1 4

New element to be inserted is 12


Step 1: Insert the new element at the end.
9
/ \
5 3
/\ /
1 4 12

Step 2: Heapify the new element following bottom-up approach.


Final heap
12
/ \
5 9
/\ /
1 4 3
Pseudo Code
void up_heapify(int heap[], int child)
{
parent = (child-1)/2;

if(heap[parent] < heap[child]) {


swap(heap[parent], heap[child])
up_heapify(heap,parent)
}

}
void insert(int heap[],int size,int key)
{
heap.append(key)
up_heapify(heap,size+1,size)
}

Deletion

The deletion operations follow the following step:

 Replace the element to be deleted by the last element in the heap.


 Delete the last item from the heap.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

 Now, the last element is placed at some position in heap, it may not follow the property of the
heap, so we need to perform down_heapify() operation in order to maintain heap structure. The
down_heapify() operation does the heapify in the top-bottom approach.

The standard deletion on Heap is to delete the element present at the root node of the heap.

Initially the heap is(It follows max-heap property)


12
/ \
6 3
/\
1 4
Element to be deleted is 12

Step 1: Replace the last element with root, and delete it.
4
/ \
6 3
/
1

Step 2: Heapify root.


Final Heap:
6
/ \
4 3
/
1
Pseudo-Code
void down_heapify(int heap[], int parent, int size)
{
largest = parent
leftChild = 2*parent + 1
rightChild = 2*parent + 2

if(leftChild < size && heap[leftChild] > heap[largest])


largest = leftChild
if(rightChild < size && heap[rightChild] > heap[largest])
largest = rightChild
if(parent != largest)
{
swap(heap[parent], heap[largest])
down_heapify(heap,largest,size)
}
}
void deleteRoot(int heap[], int size)
{
swap(heap[0], heap[size-1]) // swap first and last element

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

heap.pop_back(); // delete the last element


down_heapify(heap,0,size-1)
}

Find-max (or Find-min)

The maximum element and the minimum element in the max-heap and min-heap is
found at the root node of the heap.

int findMax(int heap[])


{
return heap[0]
}

Extract Min-Max

This operation returns and deletes the maximum or minimum element in max-heap and
min-heap respectively. The maximum element is found at the root node.

int extractMax(int heap[], int size)


{
ans = heap[0]
deleteRoot(heap, size)
return ans
}
2.9. Bounding the maximum degree.

Bounding the maximum degree: To prove that the amortized time of FIB-HEAP-
EXTRACT-MIN and FIB-HEAP-DELETE is O(lg n), we must show that the upper bound D(n)
on the degree of any node of an n-node Fibonacci heap is O(lg n). The cuts that occur in FIB-
HEAP-DECREASE-KEY, however, may cause trees within the Fibonacci heap to violate the
unordered binomial tree properties. In this section, we shall show that because we cut a node
from its parent as soon as it loses two children, D(n) is O(lg n). In particular, we shall show that
D(n) ≤ ⌊logφn⌋, where .

The key to the analysis is as follows. For each node x within a Fibonacci heap, define
size(x) to be the number of nodes, including x itself, in the subtree rooted at x. (Note that x need
not be in the root list-it can be any node at all.) We shall show that size(x) is exponential in
degree[x]. Bear in mind that degree[x] is always maintained as an accurate count of the degree
of x.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Lemma 20.1

Let x be any node in a Fibonacci heap, and suppose that degree[x] = k. Let y1, y2, . . . , yk denote
the children of x in the order in which they were linked to x, from the earliest to the latest. Then,
degree[y1] ≥ 0 and degree[yi] ≥ i - 2 for i = 2, 3, . . . , k.

Proof Obviously, degree[y1] ≥ 0.

For i ≥ 2, we note that when yi was linked to x, all of y1, y2, . . . , yi-1 were children of x, so we
must have had degree[x] = i - 1. Node yi is linked to x only if degree[x] = degree[yi], so we must
have also had degree[yi] = i - 1 at that time. Since then, node yi has lost at most one child, since
it would have been cut from x if it had lost two children. We conclude that degree[yi ] ≥ i - 2.

We finally come to the part of the analysis that explains the name "Fibonacci heaps." Recall
from Standard notations and common functions that for k = 0, 1, 2, . . . , the kth Fibonacci
number is defined by the recurrence

The following lemma gives another way to express Fk.

Lemma 20.2

For all integers k ≥ 0,

Proof The proof is by induction on k. When k = 0,

We now assume the inductive hypothesis that , and we have

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

The following lemma and its corollary complete the analysis. They use the in-equality

Fk 2 ≥ φk,

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

UNIT III GRAPHS 9


Elementary Graph Algorithms: Representations of Graphs – Breadth-First Search – Depth-First Search –
Topological Sort – Strongly Connected Components- Minimum Spanning Trees: Growing a Minimum
Spanning Tree – Kruskal and Prim- Single-Source Shortest Paths: The Bellman-Ford algorithm – Single-
Source Shortest paths in Directed Acyclic Graphs – Dijkstra‘s Algorithm; Dynamic Programming - All-
Pairs Shortest Paths: Shortest Paths and Matrix Multiplication – The Floyd-Warshall Algorithm

3.1. Elementary Graph Algorithms

There are two standard ways to represent a graph G = (V, E): as a collection of adjacency lists or
as an adjacency matrix. The adjacency-list representation is usually preferred, because it provides a
compact way to represent sparse graphs--those for which |E| is much less than |V|2. Most of the graph
algorithms presented in this book assume that an input graph is represented in adjacency-list form. An
adjacency-matrix representation may be preferred, however, when the graph is dense--|E| is close to
|V|2 -- or when we need to be able to tell quickly if there is an edge connecting two given vertices. For
example, two of the all-pairs shortest-paths algorithms presented in Chapter 26 assume that their input
graphs are represented by adjacency matrices.

The adjacency-list representation of a graph G = (V, E) consists of an array Adj of |V| lists, one
for each vertex in V. For each u V, the adjacency list Adj[u] contains (pointers to) all the vertices v
such that there is an edge (u,v) E. That is, Adj[u] consists of all the vertices adjacent to u in G. The
vertices in each adjacency list are typically stored in an arbitrary order. Figure 23.1(b) is an adjacency-
list representation of the undirected graph in Figure 23.1(a). Similarly, Figure 23.2(b) is an adjacency-
list representation of the directed graph in Figure 23.2(a).

Figure 23.1 Two representations of an undirected graph. (a) An undirected graph G having five
vertices and seven edges. (b) An adjacency-list representation of G. (c) The adjacency-matrix
representation of G.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Figure 23.2 Two representations of a directed graph. (a) A directed graph G having six vertices
and eight edges. (b) An adjacency-list representation of G. (c) The adjacency-matrix
representation of G.

If G is a directed graph, the sum of the lengths of all the adjacency lists is |E|, since an edge of
the form (u,v) is represented by having v appear in Adj[u]. If G is an undirected graph, the sum of the
lengths of all the adjacency lists is 2|E|, since if (u,v) is an undirected edge, then u appears in v's
adjacency list and vice versa. Whether a graph is directed or not, the adjacency-list representation has
the desirable property that the amount of memory it requires is O(max(V, E)) = O(V + E).

Adjacency lists can readily be adapted to represent weighted graphs, that is, graphs for which
each edge has an associated weight, typically given by a weight function w : E R. For example, let G
= (V, E) be a weighted graph with weight function w. The weight w(u,v) of the edge (u,v) E is simply
stored with vertex v in u's adjacency list. The adjacency-list representation is quite robust in that it can
be modified to support many other graph variants.

A potential disadvantage of the adjacency-list representation is that there is no quicker way to


determine if a given edge (u,v) is present in the graph than to search for v in the adjacency list Adj[u].
This disadvantage can be remedied by an adjacency-matrix re presentation of the graph, at the cost of
using asymptotically more memory.

For the adjacency-matrix representation of a graph G = (V, E), we assume that the vertices are
numbered 1, 2, . . . , |V| in some arbitrary manner. The adjacency-matrix representation of a graph G
then consists of a |V| |V| matrix A = (aij) such that

Figures 23.1(c) and 23.2(c) are the adjacency matrices of the undirected and directed graphs in
Figures 23.1(a) and 23.2(a), respectively. The adjacency matrix of a graph requires (V2) memory,
independent of the number of edges in the graph.

Observe the symmetry along the main diagonal of the adjacency matrix in Figure 23.1(c). We
define the the transpose of a matrix A = (aij) to be the matrix given by . Since in
an undirected graph, (u,v) and (v,u) represent the same edge, the adjacency matrix A of an undirected
graph is its own transpose: A = AT. In some applications, it pays to store only the entries on and above

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

the diagonal of the adjacency matrix, thereby cutting the memory needed to store the graph almost in
half.

Like the adjacency-list representation of a graph, the adjacency-matrix representation can be


used for weighted graphs. For example, if G = (V, E) is a weighted graph with edge-weight function w,
the weight w(u, v) of the edge (u,v) E is simply stored as the entry in row u and column v of the
adjacency matrix. If an edge does not exist, a NIL value can be stored as its corresponding matrix entry,
though for many problems it is convenient to use a value such as 0 or .

Although the adjacency-list representation is asymptotically at least as efficient as the


adjacency-matrix representation, the simplicity of an adjacency matrix may make it preferable when
graphs are reasonably small. Moreover, if the graph is unweighted, there is an additional advantage in
storage for the adjacency-matrix representation. Rather than using one word of computer memory for
each matrix entry, the adjacency matrix uses only one bit per entry.

3.2. Representations of Graphs – Breadth-First Search – Depth-First Search

Breadth-First Traversal (or Search) for a graph is similar to Breadth-First Traversal of a tree (See
method 2 of this post). The only catch here is, unlike trees, graphs may contain cycles, so we may
come to the same node again. To avoid processing a node more than once, we use a boolean visited
array. For simplicity, it is assumed that all vertices are reachable from the starting vertex.

For example, in the following graph, we start traversal from vertex 2. When we come to vertex 0,
we look for all adjacent vertices of it. 2 is also an adjacent vertex of 0. If we don’t mark visited
vertices, then 2 will be processed again and it will become a non-terminating process. A Breadth-First
Traversal of the following graph is 2, 0, 3, 1.

Following are the implementations of simple Breadth-First Traversal from a given source.
The implementation uses an adjacency list representation of graphs. STL‘s list container is used to

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

store lists of adjacent nodes and the queue of nodes needed for BFS traversal.

// Program to print BFS traversal from a given


// source vertex. BFS(int s) traverses vertices
// reachable from s.
#include<iostream>
#include <list>

using namespace std;

// This class represents a directed graph using


// adjacency list representation
class Graph
{
int V; // No. of vertices

// Pointer to an array containing adjacency


// lists
list<int> *adj;
public:
Graph(int V); // Constructor

// function to add an edge to graph


void addEdge(int v, int w);

// prints BFS traversal from a given source s


void BFS(int s);
};

Graph::Graph(int V)
{
this->V = V;
adj = new list<int>[V];
}

void Graph::addEdge(int v, int w)


{
adj[v].push_back(w); // Add w to v’s list.
}

void Graph::BFS(int s)
{
// Mark all the vertices as not visited
bool *visited = new bool[V];
for(int i = 0; i < V; i++)
visited[i] = false;

// Create a queue for BFS


list<int> queue;

// Mark the current node as visited and enqueue it


visited[s] = true;
queue.push_back(s);

// 'i' will be used to get all adjacent

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

// vertices of a vertex
list<int>::iterator i;

while(!queue.empty())
{
// Dequeue a vertex from queue and print it
s = queue.front();
cout << s << " ";
queue.pop_front();

// Get all adjacent vertices of the dequeued


// vertex s. If a adjacent has not been visited,
// then mark it visited and enqueue it
for (i = adj[s].begin(); i != adj[s].end(); ++i)
{
if (!visited[*i])
{
visited[*i] = true;
queue.push_back(*i);
}
}
}
}

// Driver program to test methods of graph class


int main()
{
// Create a graph given in the above diagram
Graph g(4);
g.addEdge(0, 1);
g.addEdge(0, 2);
g.addEdge(1, 2);
g.addEdge(2, 0);
g.addEdge(2, 3);
g.addEdge(3, 3);

cout << "Following is Breadth First Traversal "


<< "(starting from vertex 2) \n";
g.BFS(2);

return 0;
}

Output:

Following is Breadth First Traversal (starting from vertex 2)


2 0 3 1

Illustration :

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Note that the above code traverses only the vertices reachable from a given source vertex. All the
vertices may not be reachable from a given vertex (for example Disconnected graph). To print all the
vertices, we can modify the BFS function to do traversal starting from all nodes one by one (Like the
DFS modified version).

Time Complexity: O(V+E) where V is a number of vertices in the graph and E is a number of edges in
the graph.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Depth First Traversal (or Search) for a graph is similar to Depth First Traversal of a tree. The only
catch here is, unlike trees, graphs may contain cycles, a node may be visited twice. To avoid
processing a node more than once, use a boolean visited array.

Example:

Attention reader! Don’t stop learning now. Get hold of all the important DSA concepts with the
DSA Self Paced Course at a student-friendly price and become industry ready. To complete your
preparation from learning a language to DS Algo and many more, please refer Complete Interview
Preparation Course.

In case you wish to attend live classes with experts, please refer DSA Live Classes for Working
Professionals and Competitive Programming Live for Students.

Input: n = 4, e = 6
0 -> 1, 0 -> 2, 1 -> 2, 2 -> 0, 2 -> 3, 3 -> 3
Output: DFS from vertex 1 : 1 2 0 3
Explanation:
DFS Diagram:

Input: n = 4, e = 6
2 -> 0, 0 -> 2, 1 -> 2, 0 -> 1, 3 -> 3, 1 -> 3
Output: DFS from vertex 2 : 2 0 1 3
Explanation:
DFS Diagram:

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Solution:

 Approach: Depth-first search is an algorithm for traversing or searching tree or graph data
structures. The algorithm starts at the root node (selecting some arbitrary node as the root
node in the case of a graph) and explores as far as possible along each branch before
backtracking. So the basic idea is to start from the root or any arbitrary node and mark the
node and move to the adjacent unmarked node and continue this loop until there is no
unmarked adjacent node. Then backtrack and check for other unmarked nodes and traverse
them. Finally, print the nodes in the path.
 Algorithm:

 Create a recursive function that takes the index of the node and a visited array.
 Mark the current node as visited and print the node.
 Traverse all the adjacent and unmarked nodes and call the recursive function with
the index of the adjacent node.

Implementation:

// C++ program to print DFS traversal from


// a given vertex in a given graph
#include <bits/stdc++.h>
using namespace std;

// Graph class represents a directed graph


// using adjacency list representation

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

class Graph
{
public:
map<int, bool> visited;
map<int, list<int>> adj;

// function to add an edge to graph


void addEdge(int v, int w);

// DFS traversal of the vertices


// reachable from v
void DFS(int v);
};

void Graph::addEdge(int v, int w)


{
adj[v].push_back(w); // Add w to v’s list.
}

void Graph::DFS(int v)
{
// Mark the current node as visited and
// print it
visited[v] = true;
cout << v << " ";

// Recur for all the vertices adjacent


// to this vertex
list<int>::iterator i;
for (i = adj[v].begin(); i != adj[v].end();
++i)
if (!visited[*i])
DFS(*i);
}

// Driver code
int main()
{
// Create a graph given in the above diagram
Graph g;
g.addEdge(0, 1);
g.addEdge(0, 2);
g.addEdge(1, 2);
g.addEdge(2, 0);
g.addEdge(2, 3);
g.addEdge(3, 3);

cout << "Following is Depth First Traversal"


" (starting from vertex 2) \n";
g.DFS(2);

return 0;
}

// improved by Vishnudev C

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Output:

Following is Depth First Traversal (starting from vertex 2)


2 0 1 3

Complexity Analysis:

 Time complexity: O(V + E), where V is the number of vertices and E is the number of edges in the
graph.
 Space Complexity: O(V).
Since an extra visited array is needed of size V.

Handling Disconnected Graph

 Solution: This will happen by handling a corner case.


The above code traverses only the vertices reachable from a given source vertex. All the
vertices may not be reachable from a given vertex, as in a Disconnected graph. To do a
complete DFS traversal of such graphs, run DFS from all unvisited nodes after a DFS.
The recursive function remains the same.
 Algorithm:
o Create a recursive function that takes the index of the node and a visited array.
o Mark the current node as visited and print the node.
o Traverse all the adjacent and unmarked nodes and call the recursive function with
the index of the adjacent node.
o Run a loop from 0 to the number of vertices and check if the node is unvisited in the
previous DFS, call the recursive function with the current node.

Implementation:

// C++ program to print DFS


// traversal for a given given
// graph
#include <bits/stdc++.h>
using namespace std;

class Graph {

// A function used by DFS


void DFSUtil(int v);

public:
map<int, bool> visited;
map<int, list<int>> adj;
// function to add an edge to graph
void addEdge(int v, int w);

// prints DFS traversal of the complete graph


void DFS();
};

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

void Graph::addEdge(int v, int w)


{
adj[v].push_back(w); // Add w to v’s list.
}

void Graph::DFSUtil(int v)
{
// Mark the current node as visited and print it
visited[v] = true;
cout << v << " ";

// Recur for all the vertices adjacent to this vertex


list<int>::iterator i;
for (i = adj[v].begin(); i != adj[v].end(); ++i)
if (!visited[*i])
DFSUtil(*i);
}

// The function to do DFS traversal. It uses recursive


// DFSUtil()
void Graph::DFS()
{
// Call the recursive helper function to print DFS
// traversal starting from all vertices one by one
for (auto i:adj)
if (visited[i.first] == false)
DFSUtil(i.first);
}

// Driver Code
int main()
{
// Create a graph given in the above diagram
Graph g;
g.addEdge(0, 1);
g.addEdge(0, 9);
g.addEdge(1, 2);
g.addEdge(2, 0);
g.addEdge(2, 3);
g.addEdge(9, 3);

cout << "Following is Depth First Traversal \n";


g.DFS();

return 0;
}
//improved by Vishnudev C

Output:

Following is Depth First Traversal


0 1 2 3 9

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Complexity Analysis:

 Time complexity: O(V + E), where V is the number of vertices and E is the number of edges
in the graph.
 Space Complexity :O(V).
Since an extra visited array is needed of size V.

3.3.Topological Sort – Strongly Connected Components

A topological sort on a Graph G(V,E) is a arrangement of its nodes so that all the edges point from
left to right. This can be useful in cases where say there is a ordering of a certain set of events and such
events are repressented by a directed edge from node u to v if u occurs before v. Now if we want to
have an arrangement of node so that they are ordered as written above we use the following algorithm :

Topological Sort(G)

1. Run DFS on the Grpah

2. Whenever a node is "finished" immediately place it in a linked list.

3. return the list

This algo works because a node which is first finished is put into the list and the next finshed node

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

is put in next. The last node to finish will be on top of the list and this will be the left most event.

Strongly Connected Components

A strongly connected component of a digraph is a MAXIMAL set of vertices C such that for every
pair of vertices there is a path from u to v and from v to u.

We can make use of DFS to find the SCCs of a digraph using the following algortihm

SCC(G)

1.Run DFS on the graph

2. Compute Gt(the transpose graph)

3.Run DFS on Gt in order of decreasing finish time

4. Return the trees formed in DFS

3.4. Minimum Spanning Trees: Growing a Minimum Spanning Tree

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

What is a Spanning Tree?

Given an undirected and connected graph , a spanning tree of the graph is a tree that spans (that is,
it includes every vertex of ) and is a subgraph of (every edge in the tree belongs to ).

What is a Minimum Spanning Tree?

The cost of the spanning tree is the sum of the weights of all the edges in the tree. There can be
many spanning trees. Minimum spanning tree is the spanning tree where the cost is minimum among
all the spanning trees. There also can be many minimum spanning trees.

Minimum spanning tree has direct application in the design of networks. It is used in algorithms
approximating the travelling salesman problem, multi-terminal minimum cut problem and minimum-
cost weighted perfect matching. Other practical applications are:

1. Cluster Analysis
2. Handwriting recognition
3. Image segmentation

3.5. Kruskal and Prim- Single-Source Shortest Paths

Kruskal’s Algorithm:-

Kruskal’s Algorithm builds the spanning tree by adding edges one by one into a growing spanning
tree. Kruskal's algorithm follows greedy approach as in each iteration it finds an edge which has least
weight and add it to the growing spanning tree.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Algorithm Steps:

 Sort the graph edges with respect to their weights.


 Start adding edges to the MST from the edge with the smallest weight until the edge of the largest
weight.
 Only add edges which doesn't form a cycle , edges which connect only disconnected components.

So now the question is how to check if vertices are connected or not ?

This could be done using DFS which starts from the first vertex, then check if the second vertex is
visited or not. But DFS will make time complexity large as it has an order of where is the number of
vertices, is the number of edges. So the best solution is "Disjoint Sets":
Disjoint sets are sets whose intersection is the empty set so it means that they don't have any element in
common.

Consider following example:

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

In Kruskal’s algorithm, at each iteration we will select the edge with the lowest weight. So, we
will start with the lowest weighted edge first i.e., the edges with weight 1. After that we will select the
second lowest weighted edge i.e., edge with weight 2. Notice these two edges are totally disjoint. Now,
the next edge will be the third lowest weighted edge i.e., edge with weight 3, which connects the two
disjoint pieces of the graph. Now, we are not allowed to pick the edge with weight 4, that will create a
cycle and we can’t have any cycles. So we will select the fifth lowest weighted edge i.e., edge with
weight 5. Now the other two edges will create cycles so we will ignore them. In the end, we end up
with a minimum spanning tree with total cost 11 ( = 1 + 2 + 3 + 5).

Implementation:

#include <iostream>
#include <vector>
#include <utility>
#include <algorithm>

using namespace std;


const int MAX = 1e4 + 5;
int id[MAX], nodes, edges;
pair <long long, pair<int, int> > p[MAX];

void initialize()
{
for(int i = 0;i < MAX;++i)
id[i] = i;
}

int root(int x)
{
while(id[x] != x)
{
id[x] = id[id[x]];
x = id[x];
}
return x;
}

void union1(int x, int y)


{
int p = root(x);
int q = root(y);
id[p] = id[q];
}

long long kruskal(pair<long long, pair<int, int> > p[])


{
int x, y;
long long cost, minimumCost = 0;
for(int i = 0;i < edges;++i)
{

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

// Selecting edges one by one in increasing order from the beginning


x = p[i].second.first;
y = p[i].second.second;
cost = p[i].first;
// Check if the selected edge is creating a cycle or not
if(root(x) != root(y))
{
minimumCost += cost;
union1(x, y);
}
}
return minimumCost;
}

int main()
{
int x, y;
long long weight, cost, minimumCost;
initialize();
cin >> nodes >> edges;
for(int i = 0;i < edges;++i)
{
cin >> x >> y >> weight;
p[i] = make_pair(weight, make_pair(x, y));
}
// Sort the edges in the ascending order
sort(p, p + edges);
minimumCost = kruskal(p);
cout << minimumCost << endl;
return 0;
}

Time Complexity: In Kruskal’s algorithm, most time consuming operation is sorting because the total
complexity of the Disjoint-Set operations will be

Prim’s Algorithm

Prim’s Algorithm also use Greedy approach to find the minimum spanning tree. In Prim’s
Algorithm we grow the spanning tree from a starting position. Unlike an edge in Kruskal's, we add
vertex to the growing spanning tree in Prim's.

Algorithm Steps:

 Maintain two disjoint sets of vertices. One containing vertices that are in the growing spanning tree and
other that are not in the growing spanning tree.
 Select the cheapest vertex that is connected to the growing spanning tree and is not in the growing
spanning tree and add it into the growing spanning tree. This can be done using Priority Queues. Insert
the vertices, that are connected to growing spanning tree, into the Priority Queue.
 Check for cycles. To do that, mark the nodes which have been already selected and insert only those
nodes in the Priority Queue that are not marked.

Consider the example below:

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

In Prim’s Algorithm, we will start with an arbitrary node (it doesn’t matter which one) and
mark it. In each iteration we will mark a new vertex that is adjacent to the one that we have already
marked. As a greedy algorithm, Prim’s algorithm will select the cheapest edge and mark the vertex. So
we will simply choose the edge with weight 1. In the next iteration we have three options, edges with
weight 2, 3 and 4. So, we will select the edge with weight 2 and mark the vertex. Now again we have
three options, edges with weight 3, 4 and 5. But we can’t choose edge with weight 3 as it is creating a
cycle. So we will select the edge with weight 4 and we end up with the minimum spanning tree of total
cost 7 ( = 1 + 2 +4).

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Implementation:

#include <iostream>
#include <vector>
#include <queue>
#include <functional>
#include <utility>

using namespace std;


const int MAX = 1e4 + 5;
typedef pair<long long, int> PII;
bool marked[MAX];
vector <PII> adj[MAX];

long long prim(int x)


{
priority_queue<PII, vector<PII>, greater<PII> > Q;
int y;
long long minimumCost = 0;
PII p;
Q.push(make_pair(0, x));
while(!Q.empty())
{
// Select the edge with minimum weight
p = Q.top();
Q.pop();
x = p.second;
// Checking for cycle
if(marked[x] == true)
continue;
minimumCost += p.first;
marked[x] = true;
for(int i = 0;i < adj[x].size();++i)
{
y = adj[x][i].second;
if(marked[y] == false)
Q.push(adj[x][i]);
}
}
return minimumCost;
}

int main()
{
int nodes, edges, x, y;
long long weight, minimumCost;
cin >> nodes >> edges;
for(int i = 0;i < edges;++i)
{
cin >> x >> y >> weight;
adj[x].push_back(make_pair(weight, y));
adj[y].push_back(make_pair(weight, x));
}
// Selecting 1 as the starting node
minimumCost = prim(1);
cout << minimumCost << endl;

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

return 0;
}

3.6. The Bellman-Ford algorithm – Single-Source Shortest paths in Directed Acyclic Graphs

Given a graph and a source vertex src in graph, find shortest paths from src to all vertices in the
given graph.

The graph may contain negative weight edges. We have discussed Dijkstra’s algorithm for this
problem. Dijkstra’s algorithm is a Greedy algorithm and time complexity is O(V+E LogV) (with the
use of Fibonacci heap). Dijkstra doesn’t work for Graphs with negative weight cycle, Bellman-Ford
works for such graphs. Bellman-Ford is also simpler than Dijkstra and suites well for distributed
systems. But time complexity of Bellman-Ford is O(VE), which is more than Dijkstra.

Algorithm
Following are the detailed steps.
Input: Graph and a source vertex src
Output: Shortest distance to all vertices from src. If there is a negative weight cycle, then shortest
distances are not calculated, negative weight cycle is reported.
1) This step initializes distances from the source to all vertices as infinite and distance to the source
itself as 0. Create an array dist[] of size |V| with all values as infinite except dist[src] where src is
source vertex.
2) This step calculates shortest distances. Do following |V|-1 times where |V| is the number of vertices
in given graph.
…..a) Do following for each edge u-v
………………If dist[v] > dist[u] + weight of edge uv, then update dist[v]
………………….dist[v] = dist[u] + weight of edge uv
3) This step reports if there is a negative weight cycle in graph. Do following for each edge u-v
……If dist[v] > dist[u] + weight of edge uv, then “Graph contains negative weight cycle” The idea of
step 3 is, step 2 guarantees the shortest distances if the graph doesn’t contain a negative weight cycle.
If we iterate through all edges one more time and get a shorter path for any vertex, then there is a
negative weight cycle.

How does this work? Like other Dynamic Programming Problems, the algorithm calculates shortest
paths in a bottom-up manner. It first calculates the shortest distances which have at-most one edge in
the path. Then, it calculates the shortest paths with at-most 2 edges, and so on. After the i-th iteration
of the outer loop, the shortest paths with at most i edges are calculated. There can be maximum |V| – 1
edges in any simple path, that is why the outer loop runs |v| – 1 times. The idea is, assuming that there
is no negative weight cycle, if we have calculated shortest paths with at most i edges, then an iteration
over all edges guarantees to give shortest path with at-most (i+1) edges.

Example :-

Let us understand the algorithm with following example graph. The images are taken from this source.
Let the given source vertex be 0. Initialize all distances as infinite, except the distance to the source

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

itself. Total number of vertices in the graph is 5, so all edges must be processed 4 times.
Attention reader! Don’t stop learning now. Get hold of all the important DSA concepts with the DSA
Self Paced Course at a student-friendly price and become industry ready. To complete your
preparation from learning a language to DS Algo and many more, please refer Complete Interview
Preparation Course.

In case you wish to attend live classes with experts, please refer DSA Live Classes for Working
Professionals and Competitive Programming Live for Students.

Let all edges are processed in the following order: (B, E), (D, B), (B, D), (A, B), (A, C), (D, C), (B, C),
(E, D). We get the following distances when all edges are processed the first time. The first row shows
initial distances. The second row shows distances when edges (B, E), (D, B), (B, D) and (A, B) are
processed. The third row shows distances when (A, C) is processed. The fourth row shows when (D,
C), (B, C) and (E, D) are processed.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

The first iteration guarantees to give all shortest paths which are at most 1 edge long. We get the
following distances when all edges are processed second time (The last row shows final values).

The second iteration guarantees to give all shortest paths which are at most 2 edges long. The
algorithm processes all edges 2 more times. The distances are minimized after the second iteration, so

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

third and fourth iterations don’t update the distances.

Implementation:

// A C++ program for Bellman-Ford's single source


// shortest path algorithm.
#include <bits/stdc++.h>

// a structure to represent a weighted edge in graph


struct Edge {
int src, dest, weight;
};

// a structure to represent a connected, directed and


// weighted graph
struct Graph {
// V-> Number of vertices, E-> Number of edges
int V, E;

// graph is represented as an array of edges.


struct Edge* edge;
};

// Creates a graph with V vertices and E edges


struct Graph* createGraph(int V, int E)
{
struct Graph* graph = new Graph;
graph->V = V;
graph->E = E;
graph->edge = new Edge[E];
return graph;
}

// A utility function used to print the solution


void printArr(int dist[], int n)
{
printf("Vertex Distance from Source\n");
for (int i = 0; i < n; ++i)
printf("%d \t\t %d\n", i, dist[i]);
}

// The main function that finds shortest distances from src to


// all other vertices using Bellman-Ford algorithm. The function
// also detects negative weight cycle
void BellmanFord(struct Graph* graph, int src)
{
int V = graph->V;
int E = graph->E;
int dist[V];

// Step 1: Initialize distances from src to all other vertices


// as INFINITE

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

for (int i = 0; i < V; i++)


dist[i] = INT_MAX;
dist[src] = 0;

// Step 2: Relax all edges |V| - 1 times. A simple shortest


// path from src to any other vertex can have at-most |V| - 1
// edges
for (int i = 1; i <= V - 1; i++) {
for (int j = 0; j < E; j++) {
int u = graph->edge[j].src;
int v = graph->edge[j].dest;
int weight = graph->edge[j].weight;
if (dist[u] != INT_MAX && dist[u] + weight < dist[v])
dist[v] = dist[u] + weight;
}
}

// Step 3: check for negative-weight cycles. The above step


// guarantees shortest distances if graph doesn't contain
// negative weight cycle. If we get a shorter path, then there
// is a cycle.
for (int i = 0; i < E; i++) {
int u = graph->edge[i].src;
int v = graph->edge[i].dest;
int weight = graph->edge[i].weight;
if (dist[u] != INT_MAX && dist[u] + weight < dist[v]) {
printf("Graph contains negative weight cycle");
return; // If negative cycle is detected, simply return
}
}

printArr(dist, V);

return;
}

// Driver program to test above functions


int main()
{
/* Let us create the graph given in above example */
int V = 5; // Number of vertices in graph
int E = 8; // Number of edges in graph
struct Graph* graph = createGraph(V, E);

// add edge 0-1 (or A-B in above figure)


graph->edge[0].src = 0;
graph->edge[0].dest = 1;
graph->edge[0].weight = -1;

// add edge 0-2 (or A-C in above figure)


graph->edge[1].src = 0;
graph->edge[1].dest = 2;
graph->edge[1].weight = 4;

// add edge 1-2 (or B-C in above figure)

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

graph->edge[2].src = 1;
graph->edge[2].dest = 2;
graph->edge[2].weight = 3;

// add edge 1-3 (or B-D in above figure)


graph->edge[3].src = 1;
graph->edge[3].dest = 3;
graph->edge[3].weight = 2;

// add edge 1-4 (or A-E in above figure)


graph->edge[4].src = 1;
graph->edge[4].dest = 4;
graph->edge[4].weight = 2;

// add edge 3-2 (or D-C in above figure)


graph->edge[5].src = 3;
graph->edge[5].dest = 2;
graph->edge[5].weight = 5;

// add edge 3-1 (or D-B in above figure)


graph->edge[6].src = 3;
graph->edge[6].dest = 1;
graph->edge[6].weight = 1;

// add edge 4-3 (or E-D in above figure)


graph->edge[7].src = 4;
graph->edge[7].dest = 3;
graph->edge[7].weight = -3;

BellmanFord(graph, 0);

return 0;
}

Output:

Vertex Distance from Source


0 0
1 -1
2 2
3 -2
4 1

Notes
1) Negative weights are found in various applications of graphs. For example, instead of paying cost
for a path, we may get some advantage if we follow the path.
2) Bellman-Ford works better (better than Dijkstra’s) for distributed systems. Unlike Dijkstra’s where
we need to find the minimum value of all vertices, in Bellman-Ford, edges are considered one by one.

3) Bellman-Ford does not work with undirected graph with negative edges as it will declared as
negative cycle.
Exercise

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

1) The standard Bellman-Ford algorithm reports the shortest path only if there are no negative weight
cycles. Modify it so that it reports minimum distances even if there is a negative weight cycle.

3.7. Dijkstra‘s Algorithm

Given a graph and a source vertex in the graph, find the shortest paths from the source to all
vertices in the given graph.

Dijkstra’s algorithm is very similar to Prim’s algorithm for minimum spanning tree. Like Prim’s
MST, we generate a SPT (shortest path tree) with a given source as a root. We maintain two sets, one
set contains vertices included in the shortest-path tree, other set includes vertices not yet included in
the shortest-path tree. At every step of the algorithm, we find a vertex that is in the other set (set of not
yet included) and has a minimum distance from the source.
Below are the detailed steps used in Dijkstra’s algorithm to find the shortest path from a single source
vertex to all other vertices in the given graph.

Algorithm:-
1) Create a set sptSet (shortest path tree set) that keeps track of vertices included in the shortest-path
tree, i.e., whose minimum distance from the source is calculated and finalized. Initially, this set is
empty.
2) Assign a distance value to all vertices in the input graph. Initialize all distance values as INFINITE.
Assign distance value as 0 for the source vertex so that it is picked first.
3) While sptSet doesn’t include all vertices
….a) Pick a vertex u which is not there in sptSet and has a minimum distance value.
….b) Include u to sptSet.
….c) Update distance value of all adjacent vertices of u. To update the distance values, iterate through
all adjacent vertices. For every adjacent vertex v, if the sum of distance value of u (from source) and
weight of edge u-v, is less than the distance value of v, then update the distance value of v.

Let us understand with the following example:

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

The set sptSet is initially empty and distances assigned to vertices are {0, INF, INF, INF, INF, INF,
INF, INF} where INF indicates infinite. Now pick the vertex with a minimum distance value. The
vertex 0 is picked, include it in sptSet. So sptSet becomes {0}. After including 0 to sptSet, update
distance values of its adjacent vertices. Adjacent vertices of 0 are 1 and 7. The distance values of 1 and
7 are updated as 4 and 8. The following subgraph shows vertices and their distance values, only the
vertices with finite distance values are shown. The vertices included in SPT are shown in green colour.

Pick the vertex with minimum distance value and not already included in SPT (not in sptSET).
The vertex 1 is picked and added to sptSet. So sptSet now becomes {0, 1}. Update the distance values
of adjacent vertices of 1. The distance value of vertex 2 becomes 12.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Pick the vertex with minimum distance value and not already included in SPT (not in sptSET).
Vertex 7 is picked. So sptSet now becomes {0, 1, 7}. Update the distance values of adjacent vertices of
7. The distance value of vertex 6 and 8 becomes finite (15 and 9 respectively).

Pick the vertex with minimum distance value and not already included in SPT (not in sptSET). Vertex
6 is picked. So sptSet now becomes {0, 1, 7, 6}. Update the distance values of adjacent vertices of 6.
The distance value of vertex 5 and 8 are updated.

We repeat the above steps until sptSet includes all vertices of the given graph. Finally, we get the
following Shortest Path Tree (SPT).

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

How to implement the above algorithm?

// A C++ program for Dijkstra's single source shortest path algorithm.


// The program is for adjacency matrix representation of the graph
#include <iostream>
using namespace std;
#include <limits.h>

// Number of vertices in the graph


#define V 9

// A utility function to find the vertex with minimum distance value, from
// the set of vertices not yet included in shortest path tree
int minDistance(int dist[], bool sptSet[])
{

// Initialize min value


int min = INT_MAX, min_index;

for (int v = 0; v < V; v++)


if (sptSet[v] == false && dist[v] <= min)
min = dist[v], min_index = v;

return min_index;
}

// A utility function to print the constructed distance array


void printSolution(int dist[])
{
cout <<"Vertex \t Distance from Source" << endl;
for (int i = 0; i < V; i++)
cout << i << " \t\t"<<dist[i]<< endl;
}

// Function that implements Dijkstra's single source shortest path algorithm


// for a graph represented using adjacency matrix representation
void dijkstra(int graph[V][V], int src)
{
int dist[V]; // The output array. dist[i] will hold the shortest
// distance from src to i

bool sptSet[V]; // sptSet[i] will be true if vertex i is included in


shortest
// path tree or shortest distance from src to i is finalized

// Initialize all distances as INFINITE and stpSet[] as false


for (int i = 0; i < V; i++)
dist[i] = INT_MAX, sptSet[i] = false;

// Distance of source vertex from itself is always 0


dist[src] = 0;

// Find shortest path for all vertices


for (int count = 0; count < V - 1; count++) {
// Pick the minimum distance vertex from the set of vertices not

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

// yet processed. u is always equal to src in the first iteration.


int u = minDistance(dist, sptSet);

// Mark the picked vertex as processed


sptSet[u] = true;

// Update dist value of the adjacent vertices of the picked vertex.


for (int v = 0; v < V; v++)

// Update dist[v] only if is not in sptSet, there is an edge from


// u to v, and total weight of path from src to v through u is
// smaller than current value of dist[v]
if (!sptSet[v] && graph[u][v] && dist[u] != INT_MAX
&& dist[u] + graph[u][v] < dist[v])
dist[v] = dist[u] + graph[u][v];
}

// print the constructed distance array


printSolution(dist);
}

// driver program to test above function


int main()
{

/* Let us create the example graph discussed above */


int graph[V][V] = { { 0, 4, 0, 0, 0, 0, 0, 8, 0 },
{ 4, 0, 8, 0, 0, 0, 0, 11, 0 },
{ 0, 8, 0, 7, 0, 4, 0, 0, 2 },
{ 0, 0, 7, 0, 9, 14, 0, 0, 0 },
{ 0, 0, 0, 9, 0, 10, 0, 0, 0 },
{ 0, 0, 4, 14, 10, 0, 2, 0, 0 },
{ 0, 0, 0, 0, 0, 2, 0, 1, 6 },
{ 8, 11, 0, 0, 0, 0, 1, 0, 7 },
{ 0, 0, 2, 0, 0, 0, 6, 7, 0 } };

dijkstra(graph, 0);

return 0;
}

// This code is contributed by shivanisinghss2110

Output:

Vertex Distance from Source


0 0
1 4
2 12
3 19
4 21
5 11
6 9
7 8
8 14

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

3.8. Dynamic Programming - All-Pairs Shortest Paths

Dynamic Programming is mainly an optimization over plain recursion. Wherever we see a


recursive solution that has repeated calls for same inputs, we can optimize it using Dynamic
Programming. The idea is to simply store the results of subproblems, so that we do not have to re-
compute them when needed later. This simple optimization reduces time complexities from
exponential to polynomial. For example, if we write simple recursive solution for Fibonacci Numbers,
we get exponential time complexity and if we optimize it by storing solutions of subproblems, time
complexity reduces to linear.

Dynamic programming helps to solve a complex problem by breaking it down into a collection of
simpler subproblems, solving each of those sub-problems just once, and storing their solutions. A
dynamic programming algorithm will examine the previously solved sub problems and will combine
their solutions to give the best solution for the given problem.

 Optimal substructure: optimal solution to problem consists of optimal solutions to


subproblems.
 Overlapping subproblems: few subproblems in total, many recurring instances of each.

Solve bottom-up, building a table of solved subproblems that are used to solve larger ones.

3.9. Shortest Paths and Matrix Multiplication


Given a Weighted Directed Acyclic Graph and a source vertex in the graph, find the shortest
paths from given source to all other vertices. For a general weighted graph, we can calculate single

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

source shortest distances in O(VE) time using Bellman–Ford Algorithm. For a graph with no negative
weights, we can do better and calculate single source shortest distances in O(E + VLogV) time
using Dijkstra’s algorithm. Can we do even better for Directed Acyclic Graph (DAG)? We can
calculate single source shortest distances in O(V+E) time for DAGs. The idea is to use Topological
Sorting.

We initialize distances to all vertices as infinite and distance to source as 0, then we find a topological
sorting of the graph. Topological Sorting of a graph represents a linear ordering of the graph. Once we
have topological order (or linear representation), we one by one process all vertices in topological
order. For every vertex being processed, we update distances of its adjacent using distance of current
vertex.

1) Initialize dist[] = {INF, INF, ….} and dist[s] = 0 where s is the source vertex.
2) Create a toplogical order of all vertices.
3) Do following for every vertex u in topological order.
………..Do following for every adjacent vertex v of u
………………if (dist[v] > dist[u] + weight(u, v))
………………………dist[v] = dist[u] + weight(u, v)
http://www.stoimen.com/blog/2012/10/28/computer-algorithms-shortest-path-in-a-directed-acyclic-
graph/
1. Topologically sort G into L;
2. Set the distance to the source to 0;
3. Set the distances to all other vertices to infinity;
4. For each vertex u in L
5. - Walk through all neighbors v of u;
6. - If dist(v) > dist(u) + w(u, v)
7. - Set dist(v) <- dist(u) + w(u, v);
// The function to find shortest paths from given vertex. It uses recursive
// topologicalSortUtil() to get topological sorting of given graph.
void Graph::shortestPath(int s)
{
stack<int> Stack;

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

int dist[V];
// Mark all the vertices as not visited
bool *visited = new bool[V];
for (int i = 0; i < V; i++)
visited[i] = false;
// Call the recursive helper function to store Topological Sort
// starting from all vertices one by one
for (int i = 0; i < V; i++)
if (visited[i] == false)
topologicalSortUtil(i, visited, Stack);
// Initialize distances to all vertices as infinite and distance
// to source as 0
for (int i = 0; i < V; i++)
dist[i] = INF;
dist[s] = 0;
// Process vertices in topological order
while (Stack.empty() == false)
{
// Get the next vertex from topological order
int u = Stack.top();
Stack.pop();
// Update distances of all adjacent vertices
list<AdjListNode>::iterator i;
if (dist[u] != INF)
{
for (i = adj[u].begin(); i != adj[u].end(); ++i)
if (dist[i->getV()] > dist[u] + i->getWeight())
dist[i->getV()] = dist[u] + i->getWeight();
}
}

// Print the calculated shortest distances


for (int i = 0; i < V; i++)
(dist[i] == INF)? cout << "INF ": cout << dist[i] << " ";
}
void Graph::topologicalSortUtil(int v, bool visited[], stack<int> &Stack)
{
// Mark the current node as visited
visited[v] = true;
// Recur for all the vertices adjacent to this vertex
list<AdjListNode>::iterator i;
for (i = adj[v].begin(); i != adj[v].end(); ++i)
{
AdjListNode node = *i;
if (!visited[node.getV()])
topologicalSortUtil(node.getV(), visited, Stack);

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

}
// Push current vertex to stack which stores topological sort
Stack.push(v);
}

Time Complexity: Time complexity of topological sorting is O(V+E). After finding topological order,
the algorithm process all vertices and for every vertex, it runs a loop for all adjacent vertices. Total
adjacent vertices in a graph is O(E). So the inner loop runs O(V+E) times. Therefore, overall time
complexity of this algorithm is O(V+E).

DAG shortest paths : Solve the single-source shortest-path problem in a weighted directed acyclic
graph by 1) doing a topological sort on the vertices by edge so vertices with no incoming edges are
first and vertices with only incoming edges are last, 2) assign an infinite distance to every vertex
(dist(v)=∞) and a zero distance to the source, and 3) for each vertex v in sorted order, for each
outgoing edge e(v,u), if dist(v) + weight(e) < dist(u), set dist(u)=dist(v) + weight(e) and the
predecessor of u to v.

3.10. The Floyd-Warshall Algorithm

The Algorithm Summarized

You are going to manage a matrix of shortest paths. Call it SP, and SP[i][j], at the end of the
algorithm, will contain the shortest path from node i to node j. You initialize SP to be the adjacency
matrix for a graph. If there is no edge from i to j, then initialize SP[i][j] to be infinity or an
appropriately high sentinel value. You should also initialize SP[i][i] to be zero.

I'll now summarize the algorithm.

 For each node i in the graph:


o For each pair of nodes x, y in the graph:
 Check to see if SP[x][i] + SP[i][y] is less than SP[x][y].
 If so, what we've discovered is a shorter path from x to y than the one we
currently know, and that path goes through node i.
 In that case, update SP[x][y] to equal SP[x][i] + SP[i][y].

That's it. This is clearly O(|V|3), which is better than running Dijkstra's shortest path algorithm from
every node, when the graph is dense. That running time would be O(|V|3log(|V|)).

An Example:-
This is the same example as in the Wikipedia page (at least as of March, 2016. If Wikipedia changes,
go ahead and use the Wayback Machine to make it match up). Here's the graph

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

You'll note first that it has negative edges. That's ok, as long as there are no negative cycles in the
graph (which there aren't). Now, we're going to work through the algorithm, and what I'll do is at each
step, show you SP both in graphical form and as a matrix. I'm going to sentinelize the lack of an edge
with ∞.

1 2 3 4
----------------------
1 | 0 ∞ -2 ∞
2 | 4 0 3 ∞
3 | ∞ ∞ 0 2
4 | ∞ -1 ∞ 0

Step 1: i = 1. We start with i = 1. We next have to iterate through every pair of nodes (x,y), and test to
see if the path from x to y through node i is shorter than SP[x][y]. Obviously, we can ignore the cases
when x=i, y=i or x=y. Here are the values that we test: I've colored the "winners" red in each case:

x y SP[x][1] SP[1][y] SP[x][1]+SP[1][y] Old SP[x][y] New SP[x][y]

2 3 4 -2 2 3 2
2 4 4 ∞ ∞ 4 4
3 2 ∞ ∞ ∞ 2 2
3 4 ∞ ∞ ∞ 4 4
4 2 ∞ ∞ ∞ 2 2
4 3 ∞ -2 ∞ 3 3

As you can see, there is one place where SP is improved. I'll update the drawing and the matrix below.
I've colored the changed edges/values in red, and I colored node 1 in green to show that it was the
intermediate node in these paths:

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

1 2 3 4
----------------------
1 | 0 ∞ -2 ∞
2 | 4 0 2 ∞
3 | ∞ ∞ 0 2
4 | ∞ -1 ∞ 0

Step 2: i = 2. Let's make the same table as before, only now with i = 2:

x y SP[x][2] SP[2][y] SP[x][2]+SP[2][y] Old SP[x][y] New SP[x][y]

1 3 ∞ 2 ∞ -2 -2
1 4 ∞ ∞ ∞ ∞ ∞
3 1 ∞ 4 ∞ ∞ ∞
3 4 ∞ ∞ ∞ 2 2
4 1 -1 4 3 ∞ 3
4 3 -1 2 1 ∞ 1

There are two places where SP is improved. I'll update the drawing and the matrix below:

1 2 3 4
----------------------
1 | 0 ∞ -2 ∞
2 | 4 0 2 ∞
3 | ∞ ∞ 0 2
4 | 3 -1 1 0

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Step 3: i = 3. Here's our table:

x y SP[x][3] SP[3][y] SP[x][3]+SP[3][y] Old SP[x][y] New SP[x][y]

1 2 -2 ∞ ∞ ∞ -2
1 4 -2 2 0 ∞ ∞
2 1 2 ∞ ∞ 4 ∞
2 4 2 2 4 ∞ 2
4 1 1 ∞ ∞ 3 3
4 2 1 ∞ ∞ -1 1

Once again there are two places where SP is improved. I'll update the drawing and the matrix below:

1 2 3 4
----------------------
1 | 0 ∞ -2 0
2 | 4 0 2 4
3 | ∞ ∞ 0 2
4 | 3 -1 1 0

The Last Step: i = 4. Here's our table:

x y SP[x][4] SP[4][y] SP[x][4]+SP[4][y] Old SP[x][y] New SP[x][y]

1 2 0 -1 -1 ∞ -1
1 3 0 1 1 -2 -2
2 1 4 3 7 4 4
2 3 4 1 5 2 2
3 1 2 3 5 ∞ 5
3 2 2 -1 1 ∞ 1

We're done -- the final drawing and matrix are below. As you can see, three values were changed, and
there are no more big values on the graph at all.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

1 2 3 4
----------------------
1 | 0 -1 -2 0
2 | 4 0 2 4
3 | 5 1 0 2
4 | 3 -1 1 0

The matrix now has your all-pairs shortest paths. If any of the diagonal entries are negative, then your
graph has negative cycles, and the matrix is invalid.

Other problems solved by Floyd-Warshall


As noted in the Wikipedia page, you can solve other problems with this technique, including the following two,
which we'll explore below:

1. Calculate the transitive closure of a binary relation. A binary relation R over a set of numbers is a
definition such that for each i and j in the set, R(i,j) equals 0 or 1. The transitive closure of R is a new
relation R' such that if R(i,j) = 1, then R'(i,j) also equals 1. Moreover, R'(i,j) is minimally transitive. The
transitive property means that if R'(i,k)=1 and R'(k,j)=1, then R'(i,j)=1. The "minimally transitive" part
means that of all possible definitions of R', we choose the one that has the maximum number of i,j pairs
where R'(i,j)=0. I know that was mathy, but we'll give it a practical instance below.
2. Calculate the maximum flow paths between every pair of nodes in a directed, weighted graph.
You can see where this problem could have practicality in traffic analysis.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

UNIT IV ALGORITHM DESIGN TECHNIQUES 9


Dynamic Programming: Matrix-Chain Multiplication – Elements of Dynamic Programming – Longest Common
ubsequence- Greedy Algorithms: – Elements of the Greedy Strategy- An Activity-Selection Problem - Huffman Coding.

4.1. Dynamic Programming: Matrix-Chain Multiplication –

Dynamic programming is basically, recursion plus using common sense. What it means
is that recursion allows you to express the value of a function in terms of other values of that
function. Where the common sense tells you that if you implement your function in a way that
the recursive calls are done in advance, and stored for easy access, it will make your program
faster. This is what we call Memoization - it is memorizing the results of some specific states,
which can then be later accessed to solve other sub-problems.

Let's try to understand this by taking an example of Fibonacci numbers.

Fibonacci (n) = 1; if n = 0
Fibonacci (n) = 1; if n = 1
Fibonacci (n) = Fibonacci(n-1) + Fibonacci(n-2)

So, the first few numbers in this series will be: 1, 1, 2, 3, 5, 8, 13, 21... and so on!

A code for it using pure recursion:

int fib (int n) {


if (n < 2)
return 1;
return fib(n-1) + fib(n-2);
}

Using Dynamic Programming approach with memoization:

void fib () {
fibresult[0] = 1;
fibresult[1] = 1;
for (int i = 2; i<n; i++)
fibresult[i] = fibresult[i-1] + fibresult[i-2];
}

1. Optimization problems.
2. Combinatorial problems.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

The optimization problems expect you to select a feasible solution, so that the value of the
required function is minimized or maximized. Combinatorial problems expect you to figure out
the number of ways to do something, or the probability of some event happening.

Every Dynamic Programming problem has a schema to be followed:

 Show that the problem can be broken down into optimal sub-problems.
 Recursively define the value of the solution by expressing it in terms of optimal
solutions for smaller sub-problems.
 Compute the value of the optimal solution in bottom-up fashion.
 Construct an optimal solution from the computed information.

Bottom up vs. Top Down:

 Bottom Up - I'm going to learn programming. Then, I will start practicing. Then, I will
start taking part in contests. Then, I'll practice even more and try to improve. After
working hard like crazy, I'll be an amazing coder.
 Top Down - I will be an amazing coder. How? I will work hard like crazy. How? I'll
practice more and try to improve. How? I'll start taking part in contests. Then? I'll
practicing. How? I'm going to learn programming.

Let's look at a sample problem:

Let us say that you are given a number N, you've to find the number of different ways to
write it as the sum of 1, 3 and 4.

For example, if N = 5, the answer would be 6.

 1+1+1+1+1
 1+4
 4+1
 1+1+3
 1+3+1
 3+1+1

Sub-problem: DPn be the number of ways to write N as the sum of 1, 3, and 4.


Finding recurrence: Consider one possible solution, n = x1 + x2 + ... xn. If the last number is
1, the sum of the remaining numbers should be n - 1. So, number of sums that end with 1 is
equal to DPn-1.. Take other cases into account where the last number is 3 and 4. The final
recurrence would be:

DPn = DPn-1 + DPn-3 + DPn-4.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Take care of the base cases. DP0 = DP1 = DP2 = 1, and DP3 = 2.

Implementation:

DP[0] = DP[1] = DP[2] = 1; DP[3] = 2;


for (i = 4; i <= n; i++) {
DP[i] = DP[i-1] + DP[i-3] + DP[i-4];
}

The technique above, takes a bottom up approach and uses memoization to not compute results
that have already been computed.

Matrix-Chain Multiplication:-

Given a sequence of matrices, find the most efficient way to multiply these matrices
together. The problem is not actually to perform the multiplications, but merely to decide in
which order to perform the multiplications. We have many options to multiply a chain of
matrices because matrix multiplication is associative. In other words, no matter how we
parenthesize the product, the result will be the same. For example, if we had four matrices A, B,
C, and D, we would have:

(ABC)D = (AB)(CD) = A(BCD) = ....

However, the order in which we parenthesize the product affects the number of simple
arithmetic operations needed to compute the product, or the efficiency. For example, suppose A
is a 10 × 30 matrix, B is a 30 × 5 matrix, and C is a 5 × 60 matrix. Then,

(AB)C = (10×30×5) + (10×5×60) = 1500 + 3000 = 4500 operations


A(BC) = (30×5×60) + (10×30×60) = 9000 + 18000 = 27000 operations.

Clearly the first parenthesization requires less number of operations.


Given an array p[] which represents the chain of matrices such that the ith matrix Ai is of
dimension p[i-1] x p[i]. We need to write a function MatrixChainOrder() that should return the
minimum number of multiplications needed to multiply the chain.

Input: p[] = {40, 20, 30, 10, 30}


Output: 26000
There are 4 matrices of dimensions 40x20, 20x30, 30x10 and 10x30. Let the input 4 matrices
be A, B, C and D. The minimum number of multiplications are obtained by putting parenthesis
in following way (A(BC))D --> 20*30*10 + 40*20*10 + 40*10*30

Input: p[] = {10, 20, 30, 40, 30}


Output: 30000

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

There are 4 matrices of dimensions 10x20, 20x30, 30x40 and 40x30. Let the input 4 matrices be
A, B, C and D. The minimum number of multiplications are obtained by putting parenthesis in
following way ((AB)C)D --> 10*20*30 + 10*30*40 + 10*40*30

Input: p[] = {10, 20, 30}


Output: 6000
There are only two matrices of dimensions 10x20 and 20x30. So there is only one way to
multiply the matrices, cost of which is 10*20*30.

4.2. Elements of Dynamic Programming –

Dynamic programming posses two important elements which are as given below:

1. Overlapping sub problem

One of the main characteristics is to split the problem into subproblem, as similar as
divide and conquer approach. The overlapping subproblem is found in that problem where
bigger problems share the same smaller problem. However unlike divide and conquer there are
many subproblems in which overlap cannot be treated distinctly or independently. Basically,
there are two ways for handling the overlapping subproblems:

a. Top down approach - It is also termed as memoization technique. In this, the problem
is broken into subproblem and these subproblems are solved and the solutions are
remembered, in case if they need to be solved in future. Which means that the values are
stored in a data structure, which will help us to reach them efficiently when the same
problem will occur during the program execution.
b. Bottom up approach - It is also termed as tabulation technique. In this, all subproblems
are needed to be solved in advance and then used to build up a solution to the larger
problem.
2. Optimal sub structure

It implies that the optimal solution can be obtained from the optimal solution of its
subproblem. So optimal substructure is simply an optimal selection among all the possible
substructures that can help to select the best structure of the same kind to exist.

Components of Dynamic programming

1. Stages: The given problem can be divided into a number of subproblems which are called
stages. A stage is a small portion of given problem.
2. States:This indicates the subproblem for which the decision has to be taken. The variables
which are used for taking a decision at every stage that is called as a state variable.
3. Decision:At every stage, there can be multiple decisions out of which one of the best decisions
should be taken. The decision taken at each stage should be optimal; this is called as a stage
decision.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

4. Optimal policy:It is a rule which determines the decision at each and every stage; a policy is
called an optimal policy if it is globally optimal. This is called as Bellman principle of
optimality.

Applications of dynamic programming

1. 0/1 knapsack problem


2. Mathematical optimization problem
3. All pair Shortest path problem
4. Reliability design problem
5. Longest common subsequence (LCS)
6. Flight control and robotics control
7. Time sharing: It schedules the job to maximize CPU usage

4.3. Longest Common Subsequence- Greedy Algorithms: –

Longest Common Subsequence:

LCS Problem Statement: Given two sequences, find the length of longest subsequence present
in both of them. A subsequence is a sequence that appears in the same relative order, but not
necessarily contiguous. For example, “abc”, “abg”, “bdf”, “aeg”, ‘”acefg”, .. etc are
subsequences of “abcdefg”.

In order to find out the complexity of brute force approach, we need to first know the
number of possible different subsequences of a string with length n, i.e., find the number of
subsequences with lengths ranging from 1,2,..n-1. Recall from theory of permutation and
combination that number of combinations with 1 element are nC1. Number of combinations with
2 elements are nC2 and so forth and so on. We know that nC0 + nC1 + nC2 + … nCn = 2n. So a
string of length n has 2n-1 different possible subsequences since we do not consider the
subsequence with length 0. This implies that the time complexity of the brute force approach
will be O(n * 2n). Note that it takes O(n) time to check if a subsequence is common to both the
strings. This time complexity can be improved using dynamic programming.

It is a classic computer science problem, the basis of diff (a file comparison program that
outputs the differences between two files), and has applications in bioinformatics.

 Examples:-
LCS for input Sequences “ABCDGH” and “AEDFHR” is “ADH” of length 3.
LCS for input Sequences “AGGTAB” and “GXTXAYB” is “GTAB” of length 4.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Greedy Algorithms:-

Greedy is an algorithmic paradigm that builds up a solution piece by piece, always choosing
the next piece that offers the most obvious and immediate benefit. So the problems where
choosing locally optimal also leads to global solution are best fit for Greedy.

For example consider the Fractional Knapsack Problem. The local optimal strategy is to
choose the item that has maximum value vs weight ratio. This strategy also leads to global
optimal solution because we allowed to take fractions of an item.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

4.4. Elements of the Greedy Strategy- An Activity-Selection Problem –

Optimal Substructure:

An optimal solution to the problem contains within it optimal solutions to sub-


problems. A' = A - {1} (greedy choice) A' can be solved again with the greedy algorithm. S' = {
i � S, si � fi }

The 0 - 1 knapsack problem:

A thief has a knapsack that holds at most W pounds. Item i : ( vi, wi ) ( v = value, w =
weight ) thief must choose items to maximize the value stolen and still fit into the knapsack.
Each item must be taken or left ( 0 - 1 ).

Fractional knapsack problem:

Both the 0 - 1 and fractional problems have the optimal substructure property:
Fractional: vi / wi is the value per pound. Clearly you take as much of the item with the greatest
value per pound. This continues until you fill the knapsack. Optimal (Greedy) algorithm takes O
( n lg n ), as we must sort on vi / wi = di.

Consider the same strategy for the 0 - 1 problem:

W = 50 lbs. (maximum knapsack capacity)

w1 = 10 v1 = 60 d1.= 6
w2 = 20 v2 = 100 d2.= 5
w3 = 30 v3 = 120 d3 = 4
were d is the value density

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Greedy approach: Take all of 1, and all of 2: v1+ v2 = 160, optimal solution is to take all of 2
and 3: v2 + v3= 220, other solution is to take all of 1 and 3 v1+ v3 = 180. All below 50 lbs.

When solving the 0 - 1 knapsack problem, empty space lowers the effective d of the load. Thus
each time an item is chosen for inclusion we must consider both

 i included
 i excluded

These are clearly overlapping sub-problems for different i's and so best solved by DP.

4.5. Huffman Coding.

Huffman coding is a lossless data compression algorithm. The idea is to assign variable-
length codes to input characters, lengths of the assigned codes are based on the frequencies of
corresponding characters. The most frequent character gets the smallest code and the least
frequent character gets the largest code.

The variable-length codes assigned to input characters are Prefix Codes, means the
codes (bit sequences) are assigned in such a way that the code assigned to one character is not
the prefix of code assigned to any other character. This is how Huffman Coding makes sure that
there is no ambiguity when decoding the generated bitstream.
Let us understand prefix codes with a counter example. Let there be four characters a, b, c and

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

d, and their corresponding variable length codes be 00, 01, 0 and 1. This coding leads to
ambiguity because code assigned to c is the prefix of codes assigned to a and b. If the
compressed bit stream is 0001, the de-compressed output may be “cccd” or “ccb” or “acd” or
“ab”.
There are mainly two major parts in Huffman Coding

1. Build a Huffman Tree from input characters.


2. Traverse the Huffman Tree and assign codes to characters.

Steps to build Huffman Tree


Input is an array of unique characters along with their frequency of occurrences and output is
Huffman Tree.

1. Create a leaf node for each unique character and build a min heap of all leaf nodes (Min
Heap is used as a priority queue. The value of frequency field is used to compare two
nodes in min heap. Initially, the least frequent character is at root)
2. Extract two nodes with the minimum frequency from the min heap.

3. Create a new internal node with a frequency equal to the sum of the two nodes
frequencies. Make the first extracted node as its left child and the other extracted node as
its right child. Add this node to the min heap.
4. Repeat steps#2 and #3 until the heap contains only one node. The remaining node is the
root node and the tree is complete.
Let us understand the algorithm with an example:

character Frequency
a 5
b 9
c 12
d 13
e 16
f 45

Step 1. Build a min heap that contains 6 nodes where each node represents root of a tree with
single node.

Step 2 Extract two minimum frequency nodes from min heap. Add a new internal node with
frequency 5 + 9 = 14.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Now min heap contains 5 nodes where 4 nodes are roots of trees with single element each, and
one heap node is root of tree with 3 elements

character Frequency
c 12
d 13
Internal Node 14
e 16
f 45

Step 3: Extract two minimum frequency nodes from heap. Add a new internal node with
frequency 12 + 13 = 25

Now min heap contains 4 nodes where 2 nodes are roots of trees with single element each, and
two heap nodes are root of tree with more than one nodes

character Frequency
Internal Node 14
e 16
Internal Node 25
f 45

Step 4: Extract two minimum frequency nodes. Add a new internal node with frequency 14 +
16 = 30

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Now min heap contains 3 nodes.

character Frequency
Internal Node 25
Internal Node 30
f 45
Step 5: Extract two minimum frequency nodes. Add a new internal node with frequency 25 +
30 = 55

Now min heap contains 2 nodes.

character Frequency
f 45
Internal Node 55

Step 6: Extract two minimum frequency nodes. Add a new internal node with frequency 45 +
55 = 100

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Now min heap contains only one node.

character Frequency
Internal Node 100

Since the heap contains only one node, the algorithm stops here.

Steps to print codes from Huffman Tree: Traverse the tree formed starting from the root.
Maintain an auxiliary array. While moving to the left child, write 0 to the array. While moving
to the right child, write 1 to the array. Print the array when a leaf node is encountered.

The codes are as follows:

character code-word
f 0
c 100
d 101
a 1100
b 1101
e 111

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

UNIT V NP COMPLETE AND NP HARD 9


NP-Completeness: Polynomial Time – Polynomial-Time Verification – NP- Completeness and
Reducibility – NP-Completeness Proofs – NP-Complete Problems.

5.1. NP-Completeness: Polynomial Time

Status of NP Complete problems is another failure story, NP complete problems are problems
whose status is unknown. No polynomial time algorithm has yet been discovered for any NP complete
problem, nor has anybody yet been able to prove that no polynomial-time algorithm exists for any of
them. The interesting part is, if any one of the NP complete problems can be solved in polynomial
time, then all of them can be solved.

P is a set of problems that can be solved by a deterministic Turing machine in Polynomial time.

NP is set of decision problems that can be solved by a Non-deterministic Turing Machine in


Polynomial time. P is subset of NP (any problem that can be solved by a deterministic machine in
polynomial time can also be solved by a non-deterministic machine in polynomial time).
Informally, NP is a set of decision problems that can be solved by a polynomial-time via a “Lucky
Algorithm”, a magical algorithm that always makes a right guess among the given set of choices .

NP-complete problems are the hardest problems in the NP set. A decision problem L is NP-complete
if:
1) L is in NP (Any given solution for NP-complete problems can be verified quickly, but there is no
efficient known solution).
2) Every problem in NP is reducible to L in polynomial time (Reduction is defined below).

A problem is NP-Hard if it follows property 2 mentioned above, doesn’t need to follow property
1. Therefore, the NP-Complete set is also a subset of the NP-Hard set.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Decision vs Optimization Problems: NP-completeness applies to the realm of decision problems. It


was set up this way because it’s easier to compare the difficulty of decision problems than that of
optimization problems. In reality, though, being able to solve a decision problem in polynomial time
will often permit us to solve the corresponding optimization problem in polynomial time (using a
polynomial number of calls to the decision problem). So, discussing the difficulty of decision problems
is often really equivalent to discussing the difficulty of optimization problems. (Source Ref 2).
For example, consider the vertex cover problem (Given a graph, find out the minimum sized vertex set
that covers all edges). It is an optimization problem. Corresponding decision problem is, given
undirected graph G and k, is there a vertex cover of size k?

What is Reduction?

Let L1 and L2 be two decision problems. Suppose algorithm A2 solves L2. That is, if y is an input
for L2 then algorithm A2 will answer Yes or No depending upon whether y belongs to L2 or not.
The idea is to find a transformation from L1 to L2 so that algorithm A2 can be part of an algorithm A1
to solve L1.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Learning reduction, in general, is very important. For example, if we have library functions to solve
certain problems and if we can reduce a new problem to one of the solved problems, we save a lot of
time. Consider the example of a problem where we have to find the minimum product path in a given
directed graph where the product of path is the multiplication of weights of edges along the path. If we
have code for Dijkstra’s algorithm to find the shortest path, we can take the log of all weights and use
Dijkstra’s algorithm to find the minimum product path rather than writing a fresh code for this new
problem.

How to prove that a given problem is NP complete?

From the definition of NP-complete, it appears impossible to prove that a problem L is NP-
Complete. By definition, it requires us to that show every problem in NP in polynomial time reducible
to L. Fortunately, there is an alternate way to prove it. The idea is to take a known NP-Complete
problem and reduce it to L. If polynomial time reduction is possible, we can prove that L is NP-
Complete by transitivity of reduction (If a NP-Complete problem is reducible to L in polynomial time,
then all problems are reducible to L in polynomial time).

What was the first problem proved as NP-Complete?


There must be some first NP-Complete problem proved by definition of NP-Complete problems.
SAT (Boolean satisfiability problem) is the first NP-Complete problem proved by Cook (See CLRS
book for proof).

It is always useful to know about NP-Completeness even for engineers. Suppose you are asked to
write an efficient algorithm to solve an extremely important problem for your company. After a lot of
thinking, you can only come up exponential time approach which is impractical. If you don’t know
about NP-Completeness, you can only say that I could not come with an efficient algorithm. If you
know about NP-Completeness and prove that the problem is NP-complete, you can proudly say that
the polynomial time solution is unlikely to exist.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

5 . 2 . Polynomial-Time Verification

Before talking about the class of NP-complete problems, it is essential to introduce the notion of a
verification algorithm. Many problems are hard to solve, but they have the property that it easy to
authenticate the solution if one is provided.

Hamiltonian cycle problem:-

Consider the Hamiltonian cycle problem. Given an undirected graph G, does G have a cycle that
visits each vertex exactly once? There is no known polynomial time algorithm for this dispute.

Note: - It means you can't build a Hamiltonian cycle in a graph with a polynomial time even if
there is no specific path is given for the Hamiltonian cycle with the particular vertex, yet you
can't verify the Hamiltonian cycle within the polynomial time

Fig: Hamiltonian Cycle

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Let us understand that a graph did have a Hamiltonian cycle. It would be easy for someone to
convince of this. They would similarly say: "the period is hv3, v7, v1....v13i.

We could then inspect the graph and check that this is indeed a legal cycle and that it visits all of
the vertices of the graph exactly once. Thus, even though we know of no efficient way to solve the
Hamiltonian cycle problem, there is a beneficial way to verify that a given cycle is indeed a
Hamiltonian cycle.

Note:-For the verification in the Polynomial-time of an undirected Hamiltonian cycle graph G.


There must be exact/specific/definite path must be given of Hamiltonian cycle then you can
verify in the polynomial time.

Definition of Certificate: - A piece of information which contains in the given path of a vertex is
known as certificate

Relation of P and NP classes

1. P contains in NP
2. P=NP

1. Observe that P contains in NP. In other words, if we can solve a problem in polynomial time,
we can indeed verify the solution in polynomial time. More formally, we do not need to see a
certificate (there is no need to specify the vertex/intermediate of the specific path) to solve the
problem; we can explain it in polynomial time anyway.
2. However, it is not known whether P = NP. It seems you can verify and produce an output of the
set of decision-based problems in NP classes in a polynomial time which is impossible because
according to the definition of NP classes you can verify the solution within the polynomial
time. So this relation can never be held.

5 . 3 . NP- Completeness and Reducibility

Reductions:

The class NP-complete (NPC) problems consist of a set of decision problems (a subset of class
NP) that no one knows how to solve efficiently. But if there were a polynomial solution for even a
single NP-complete problem, then every problem in NPC will be solvable in polynomial time. For this,
we need the concept of reductions.

Suppose there are two problems, A and B. You know that it is impossible to solve problem A in
polynomial time. You want to prove that B cannot be explained in polynomial time. We want to show
that (A ∉ P) => (B ∉ P)

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Consider an example to illustrate reduction: The following problem is well-known to be NPC:

3-color: Given a graph G, can each of its vertices be labeled with one of 3 different colors such that
two adjacent vertices do not have the same label (color).

Coloring arises in various partitioning issues where there is a constraint that two objects cannot
be assigned to the same set of partitions. The phrase "coloring" comes from the original application
which was in map drawing. Two countries that contribute a common border should be colored with
different colors.

It is well known that planar graphs can be colored (maps) with four colors. There exists a polynomial
time algorithm for this. But deciding whether this can be done with 3 colors is hard, and there is no
polynomial time algorithm for it.

Fig: Example of 3-colorable and non-3-colorable graphs.

NP- Completeness:-

A decision problem L is NP-Hard if

L' ≤p L for all L' ϵ NP.

Definition: L is NP-complete if

1. L ϵ NP and
2. L' ≤ p L for some known NP-complete problem L.' Given this formal definition, the complexity
classes are:

P: is the set of decision problems that are solvable in polynomial time.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

NP: is the set of decision problems that can be verified in polynomial time.

NP-Hard: L is NP-hard if for all L' ϵ NP, L' ≤p L. Thus if we can solve L in polynomial time, we can
solve all NP problems in polynomial time.

NP-Complete L is NP-complete if

1. L ϵ NP and
2. L is NP-hard

If any NP-complete problem is solvable in polynomial time, then every NP-Complete problem is also
solvable in polynomial time. Conversely, if we can prove that any NP-Complete problem cannot be
solved in polynomial time, every NP-Complete problem cannot be solvable in polynomial time.

5.4. NP-Completeness Proofs

To start the process of being able to prove problems are NP-complete, we need to prove just one
problem H is NP-complete. After that, to show that any problem X is NP-hard, we just need to reduce
H to X. When doing NP-completeness proofs, it is very important not to get this reduction backwards!
If we reduce candidate problem X to known hard problem H, this means that we use H as a step to
solving X. All that means is that we have found a (known) hard way to solve X. However, when we
reduce known hard problem H to candidate problem X, that means we are using X as a step to solve H.
And if we know that H is hard, that means X must also be hard (because if X were not hard, then
neither would H be hard).

So a crucial first step to getting this whole theory off the ground is finding one problem that is
NP-hard. The first proof that a problem is NP-hard (and because it is in NP, therefore NP-complete)
was done by Stephen Cook. For this feat, Cook won the first Turing award, which is the closest
Computer Science equivalent to the Nobel Prize. The “grand-daddy” NP-complete problem that Cook
used is called SATISFIABILITY (or SAT for short).

A Boolean expression is comprised of Boolean variables combined using the operators AND (⋅),
OR (+), and NOT (to negate Boolean variable x we write x¯ ¯ ¯ ). A literal is a Boolean variable or its
negation. A clause is one or more literals OR’ed together. Let E be a Boolean expression over variables
x1,x2,...,xn. The we define Conjunctive Normal Form (CNF) to be a Boolean expression written as a
series of clauses that are AND’ed together. For example,

E=(x5+x7+x8¯ ¯ ¯ ¯ ¯ +x10)⋅(x2¯ ¯ ¯ ¯ ¯ +x3)⋅(x1+x3¯ ¯ ¯ ¯ ¯ +x6) is in CNF, and has three clauses. Now we can
define the problem SAT.

Problem SATISFIABILITY (SAT)

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Input: A Boolean expression E over variables x1,x2,... in Conjunctive Normal Form.

Output: YES if there is an assignment to the variables that makes E true, NO otherwise. Cook proved
that SAT is NP-hard. Explaining Cook’s proof is beyond the scope of this course. But we can briefly
summarize it as follows. Any decision problem Fcan be recast as some language acceptance problem

L:F(I)=YES⇔L(I′)=ACCEPT.

That is, if a decision problem F yields YES on input I, then there is a language L containing string
I′ where I′ is some suitable transformation of input I. Conversely, if F would give answer NO for input
I, then I ‘s transformed version I′ is not in the language L.Turing machines are a simple model of
computation for writing programs that are language acceptors. There is a “universal” Turing machine
that can take as input a description for a Turing machine, and an input string, and return the execution
of that machine on that string. This Turing machine in turn can be cast as a Boolean expression such
that the expression is satisfiable if and only if the Turing machine yields ACCEPT for that string. Cook
used Turing machines in his proof because they are simple enough that he could develop this
transformation of Turing machines to Boolean expressions, but rich enough to be able to compute any
function that a regular computer can compute. The significance of this transformation is that any
decision problem that is performable by the Turing machine is transformable to SAT. Thus, SAT is
NP-hard.

To show that a decision problem X is NP-complete, we prove that X is in NP (normally easy, and
normally done by giving a suitable polynomial-time, non-deterministic algorithm) and then prove that
X is NP-hard. To prove that X is NP-hard, we choose a known NP-complete problem, say A. We
describe a polynomial-time transformation that takes an arbitrary instance I of A to an instance I′ of X.
We then describe a polynomial-time transformation from SLN′ to SLN such that SLN is the solution for
I. The following modules show a number of known NP-complete problems, and also some proofs that
they are NP-complete. The various proofs will link the problems together as shown here:

Figure 28.12.1: We will use this sequence of reductions for the NP Complete Proof

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

5. 5. NP-Complete Problems.
NP-complete problem, any of a class of computational problems for which no efficient solution
algorithm has been found. Many significant computer-science problems belong to this class—e.g., the
traveling salesman problem, satisfiability problems, and graph-covering problems.

So-called easy, or tractable, problems can be solved by computer algorithms that run in
polynomial time; i.e., for a problem of size n, the time or number of steps needed to find the solution is
a polynomial function of n. Algorithms for solving hard, or intractable, problems, on the other hand,
require times that are exponential functions of the problem size n. Polynomial-time algorithms are
considered to be efficient, while exponential-time algorithms are considered inefficient, because the
execution times of the latter grow much more rapidly as the problem size increases.

A problem is called NP (nondeterministic polynomial) if its solution can be guessed and verified
in polynomial time; nondeterministic means that no particular rule is followed to make the guess. If a
problem is NP and all other NP problems are polynomial-time reducible to it, the problem is NP-
complete. Thus, finding an efficient algorithm for any NP-complete problem implies that an efficient
algorithm can be found for all such problems, since any problem belonging to this class can be recast
into any other member of the class. It is not known whether any polynomial-time algorithms will ever
be found for NP-complete problems, and determining whether these problems are tractable or
intractable remains one of the most important questions in theoretical computer science. When an NP-
complete problem must be solved, one approach is to use a polynomial algorithm to approximate the
solution; the answer thus obtained will not necessarily be optimal but will be reasonably close.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

NP-Complete Problems

Following are some NP-Complete problems, for which no polynomial time algorithm is known.

 Determining whether a graph has a Hamiltonian cycle


 Determining whether a Boolean formula is satisfiable, etc.

NP-Hard Problems

The following problems are NP-Hard

 The circuit-satisfiability problem


 Set Cover
 Vertex Cover
 Travelling Salesman Problem

In this context, now we will discuss TSP is NP-Complete

TSP is NP-Complete:-

The traveling salesman problem consists of a salesman and a set of cities. The salesman has to
visit each one of the cities starting from a certain one and returning to the same city. The challenge of
the problem is that the traveling salesman wants to minimize the total length of the trip

Proof:-

To prove TSP is NP-Complete, first we have to prove that TSP belongs to NP. In TSP, we find
a tour and check that the tour contains each vertex once. Then the total cost of the edges of the tour is
calculated. Finally, we check if the cost is minimum. This can be completed in polynomial time. Thus
TSP belongs to NP.

Downloaded by Raj Kumari ([email protected])


lOMoARcPSD|33989822

Secondly, we have to prove that TSP is NP-hard. To prove this, one way is to show that Hamiltonian
cycle ≤p TSP (as we know that the Hamiltonian cycle problem is NPcomplete).

Assume G = (V, E) to be an instance of Hamiltonian cycle.

Hence, an instance of TSP is constructed. We create the complete graph G' = (V, E'), where

E′={(i,j):i,j∈Vandi≠j

Thus, the cost function is defined as follows −

t(i,j)={01if(i,j)∈Eotherwise

Now, suppose that a Hamiltonian cycle h exists in G. It is clear that the cost of each edge in h is 0
in G as each edge belongs to E. Therefore, h has a cost of 0 in G'. Thus, if graph G has a Hamiltonian
'

cycle, then graph G' has a tour of 0 cost.

Conversely, we assume that G' has a tour h' of cost at most 0. The cost of edges in E' are 0 and 1
by definition. Hence, each edge must have a cost of 0 as the cost of h' is 0. We therefore conclude that
h' contains only edges in E.

PREPARED BY
Mr.R.KARTHIKEYAN ASST.PROF/MCA

ALL THE BEST


**************

Downloaded by Raj Kumari ([email protected])

You might also like