Mc4101 Ads Notes Advance Data Structure Nodes
Mc4101 Ads Notes Advance Data Structure Nodes
Mc4101 Ads Notes Advance Data Structure Nodes
SUGGESTED ACTIVITIES:
1. Write an algorithm for Towers of Hanoi problem using recursion and analyze the
COURSE OUTCOMES:
CO1:Design data structures and algorithms to solve computing problems.
CO2:Choose and implement efficient data structures and apply them to solve problems. CO3:Design
algorithms using graph structure and various string-matching algorithms tosolve real-life
problems.
CO4: Design one’s own algorithm for an unknown problem.
CO5: Apply suitable design strategy for problem solving.
REFERENCES
1. S.Sridhar,” Design and Analysis of Algorithms”, Oxford University Press, 1st Edition,
2014.
2. Adam Drozdex, “Data Structures and algorithms in C++”, Cengage Learning, 4th Edition,
2013.
3. T.H. Cormen, C.E.Leiserson, R.L. Rivest and C.Stein, "Introduction to Algorithms",
Prentice Hall of India, 3rd Edition, 2012.
4. Mark Allen Weiss, “Data Structures and Algorithms in C++”, Pearson Education,
3rd Edition, 2009.
5. E. Horowitz, S. Sahni and S. Rajasekaran, “Fundamentals of Computer Algorithms”,
University Press, 2nd Edition, 2008.
6. Alfred V. Aho, John E. Hopcroft, Jeffrey D. Ullman, “Data Structures and Algorithms”,
Pearson Education, Reprint 2006.
Suppose computers were infinitely fast and computer memory was free. Would you have any reason to
study algorithms? The answer is yes, if for no other reason than that you would still like to demonstrate
that your solution method terminates and does so with the correct answer.
If computers were infinitely fast, any correct method for solving a problem would do. You would
probably want your implementation to be within the bounds of good software engineering practice (i.e.,
well designed and documented), but you would most often use whichever method was the easiest to
implement. Of course, computers may be fast, but they are not infinitely fast. And memory may be cheap,
but it is not free. Computing time is therefore a bounded resource, and so is space in memory. These
resources should be used wisely, and algorithms that are efficient in terms of time or space will help you
do so.
Efficiency: Algorithms devised to solve the same problem often differ dramatically in their efficiency.
These differences can be much more significant than differences due to hardware and software.
As an example, in Chapter 2, we will see two algorithms for sorting. The first, known as insertion
sort, takes time roughly equal to c1n2 to sort n items, where c1 is a constant that does not depend on n. That
is, it takes time roughly proportional to n2. The second, merge sort, takes time roughly equal to c2n lg n,
where lg n stands for log2 n and c2 is another constant that also does not depend on n. Insertion sort
usually has a smaller constant factor than merge sort, so that c1 < c2. We shall see that the constant factors
can be far less significant in the running time than the dependence on the input size n. Where merge sort
has a factor of lg n in its running time, insertion sort has a factor of n, which is much larger. Although
insertion sort is usually faster than merge sort for small input sizes, once the input size n becomes large
enough, merge sort's advantage of lg n vs. n will more than compensate for the difference in constant
factors. No matter how much smaller c1 is than c2, there will always be a crossover point beyond which
merge sort is faster.
For a concrete example, let us pit a faster computer (computer A) running insertion sort against a
slower computer (computer B) running merge sort. They each must sort an array of one million numbers.
Suppose that computer A executes one billion instructions per second and computer B executes only ten
million instructions per second, so that computer A is 100 times faster than computer B in raw computing
power. To make the difference even more dramatic, suppose that the world's craftiest programmer codes
insertion sort in machine language for computer A, and the resulting code requires 2n2 instructions to sort
n numbers. (Here, c1 = 2.) Merge sort, on the other hand, is programmed for computer B by an average
programmer using a high-level language with an inefficient compiler, with the resulting code taking 50n
lg n instructions (so that c2 = 50). To sort one million numbers, computer A takes
By using an algorithm whose running time grows more slowly, even with a poor compiler, computer B
runs 20 times faster than computer A! The advantage of merge sort is even more pronounced when we
sort ten million numbers: where insertion sort takes approximately 2.3 days, merge sort takes under 20
minutes. In general, as the problem size increases, so does the relative advantage of merge sort.
The example above shows that algorithms, like computer hardware, are a technology. Total system
performance depends on choosing efficient algorithms as much as on choosing fast hardware. Just as rapid
advances are being made in other computer technologies, they are being made in algorithms as well.
You might wonder whether algorithms are truly that important on contemporary computers in light of
other advanced technologies, such as
The answer is yes. Although there are some applications that do not explicitly require algorithmic
content at the application level (e.g., some simple web-based applications), most also require a degree of
algorithmic content on their own. For example, consider a web-based service that determines how to travel
from one location to another. (Several such services existed at the time of this writing.) Its implementation
would rely on fast hardware, a graphical user interface, wide-area networking, and also possibly on object
orientation. However, it would also require algorithms for certain operations, such as finding routes
(probably using a shortest-path algorithm), rendering maps, and interpolating addresses.
Moreover, even an application that does not require algorithmic content at the application level relies
heavily upon algorithms. Does the application rely on fast hardware? The hardware design used
algorithms. Does the application rely on graphical user interfaces? The design of any GUI relies on
algorithms. Does the application rely on networking? Routing in networks relies heavily on algorithms.
Was the application written in a language other than machine code? Then it was processed by a compiler,
interpreter, or assembler, all of which make extensive use of algorithms. Algorithms are at the core of
most technologies used in contemporary computers.
Furthermore, with the ever-increasing capacities of computers, we use them to solve larger problems
than ever before. As we saw in the above comparison between insertion sort and merge sort, it is at larger
problem sizes that the differences in efficiencies between algorithms become particularly prominent.
Generally, there is always more than one way to solve a problem in computer science with different
algorithms. Therefore, it is highly required to use a method to compare the solutions in order to judge
which one is more optimal. The method must be:
Independent of the machine and its configuration, on which the algorithm is running on.
Shows a direct correlation with the number of inputs.
Can distinguish two algorithms clearly without ambiguity.
Time Complexity: The time complexity of an algorithm quantifies the amount of time taken by an
algorithm to run as a function of the length of the input. Note that the time to run is a function of the
length of the input and not the actual execution time of the machine on which the algorithm is running on.
In order to calculate time complexity on an algorithm, it is assumed that a constant time c is taken to
execute one operation, and then the total operations for an input length on N are calculated. Consider an
example to understand the process of calculation: Suppose a problem is to find whether a pair (X, Y)
exists in an array, A of N elements whose sum is Z. The simplest idea is to consider every pair and check
if it satisfies the given condition or not.
int a[n];
for(int i = 0;i < n;i++)
cin >> a[i]
return false
Assuming that each of the operations in the computer takes approximately constant time, let it be c.
The number of lines of code executed actually depends on the value of Z. During analyses of the
algorithm, mostly the worst-case scenario is considered, i.e., when there is no pair of elements with sum
equals Z. In the worst case,
So total execution time is N*c + N*N*c + c. Now ignore the lower order terms since the lower order
terms are relatively insignificant for large input, therefore only the highest order term is taken (without
constant) which is N*N in this case. Different notations are used to describe the limiting behavior of a
function, but since the worst case is taken so big-O notation will be used to represent the time complexity.
Hence, the time complexity is O(N2) for the above algorithm. Note that the time complexity is solely
based on the number of elements in array A i.e the input length, so if the length of the array will increase
the time of execution will also increase.
Order of growth is how the time of execution depends on the length of the input. In the above example, it
is clearly evident that the time of execution quadratically depends on the length of the array. Order of
growth will help to compute the running time with ease.
Another Example: Let’s calculate the time complexity of the below algorithm:
count = 0
for (int i = N; i > 0; i /= 2)
for (int j = 0; j < i; j++)
count++;
This is a tricky case. In the first look, it seems like the complexity is O(N * log N). N for the j′s loop
and log(N) for i′s loop. But it’s wrong. Let’s see why.
The total number of times count++ will run is N + N/2 + N/4+…+1= 2 * N. So the time complexity
will be O(N).
Some general time complexities are listed below with the input range for which they are accepted in
competitive programming:
Space Complexity: The space complexity of an algorithm quantifies the amount of space taken by an
algorithm to run as a function of the length of the input. Consider an example: Suppose a problem to find
the frequency of array elements.
int freq[n];
int a[n];
Here two arrays of length N, and variable i are used in the algorithm so, the total space used is N * c +
N * c + 1 * c = 2N * c + c, where c is a unit space taken. For many inputs, constant c is insignificant, and
it can be said that the space complexity is O(N).
There is also auxiliary space, which is different from space complexity. The main difference is
where space complexity quantifies the total space used by the algorithm, auxiliary space quantifies the
extra space that is used in the algorithm apart from the given input. In the above example, the auxiliary
space is the space used by the freq[] array because that is not part of the given input. So total auxiliary
space is N * c + c which is O(N) only.
Worst Case Analysis (Usually Done) : In the worst-case analysis, we calculate the upper bound on the
running time of an algorithm. We must know the case that causes a maximum number of operations to be
executed. For Linear Search, the worst case happens when the element to be searched (x in the above
code) is not present in the array. When x is not present, the search() function compares it with all the
elements of arr[] one by one. Therefore, the worst-case time complexity of linear search would be Θ(n).
Average Case Analysis (Sometimes done) : In average case analysis, we take all possible inputs and
calculate computing time for all of the inputs. Sum all the calculated values and divide the sum by the
total number of inputs. We must know (or predict) the distribution of cases. For the linear search problem,
let us assume that all cases are uniformly distributed (including the case of x not being present in the
array). So we sum all the cases and divide the sum by (n+1). Following is the value of average-case time
complexity.
= Θ(n)
Best Case Analysis (Bogus) : In the best case analysis, we calculate the lower bound on the running time
of an algorithm. We must know the case that causes a minimum number of operations to be executed. In
the linear search problem, the best case occurs when x is present at the first location. The number of
operations in the best case is constant (not dependent on n). So time complexity in the best case would be
Θ(1) Most of the times, we do worst-case analysis to analyze algorithms. In the worst analysis, we
guarantee an upper bound on the running time of an algorithm which is good information.
The average case analysis is not easy to do in most practical cases and it is rarely done. In the average case
analysis, we must know (or predict) the mathematical distribution of all possible inputs.
The Best Case analysis is bogus. Guaranteeing a lower bound on an algorithm doesn’t provide any
information as in the worst case, an algorithm may take years to run.
For some algorithms, all the cases are asymptotically the same, i.e., there are no worst and best cases. For
example, Merge Sort. Merge Sort does Θ(nLogn) operations in all cases. Most of the other sorting
algorithms have worst and best cases. For example, in the typical implementation of Quick Sort (where
pivot is chosen as a corner element), the worst occurs when the input array is already sorted and the best
occurs when the pivot elements always divide the array into two halves. For insertion sort, the worst case
occurs when the array is reverse sorted and the best case occurs when the array is sorted in the same order
as output.
The main idea of asymptotic analysis is to have a measure of the efficiency of algorithms that don’t
depend on machine-specific constants and don’t require algorithms to be implemented and time taken by
programs to be compared. Asymptotic notations are mathematical tools to represent the time complexity
of algorithms for asymptotic analysis. The following 3 asymptotic notations are mostly used to represent
the time complexity of algorithms.
In case you wish to attend live classes with experts, please refer DSA Live Classes for Working
Professionals and Competitive Programming Live for Students.
1) Θ Notation: The theta notation bounds a function from above and below, so it defines exact asymptotic
behavior. A simple way to get the Theta notation of an expression is to drop low-order terms and ignore
leading constants.
Dropping lower order terms is always fine because there will always be a number(n) after which Θ(n 3) has
higher values than Θ(n2) irrespective of the constants involved.
The above definition means, if f(n) is theta of g(n), then the value f(n) is always between c1*g(n) and
c2*g(n) for large values of n (n >= n0). The definition of theta also requires that f(n) must be non-negative
for values of n greater than n0.
2) Big O Notation: The Big O notation defines an upper bound of an algorithm, it bounds a function only
from above. For example, consider the case of Insertion Sort. It takes linear time in the best case and
quadratic time in the worst case. We can safely say that the time complexity of Insertion sort is O(n^2).
Note that O(n^2) also covers linear time.
If we use Θ notation to represent time complexity of Insertion sort, we have to use two statements
for best and worst cases:
1. The worst-case time complexity of Insertion Sort is Θ(n^2).
2. The best case time complexity of Insertion Sort is Θ(n).
The Big O notation is useful when we only have an upper bound on the time complexity of an algorithm.
Many times we easily find an upper bound by simply looking at the algorithm.
3) Ω Notation: Just as Big O notation provides an asymptotic upper bound on a function, Ω notation
provides an asymptotic lower bound. Ω Notation can be useful when we have a lower bound on the time
complexity of an algorithm. As discussed in the previous post, the best case performance of an algorithm
is generally not useful, the Omega notation is the least used notation among all three.
Let us consider the same Insertion sort example here. The time complexity of Insertion Sort can be written
as Ω(n), but it is not very useful information about insertion sort, as we are generally interested in worst-
case and sometimes in the average case.
1. General Properties :
We can say
If f(n) is Θ(g(n)) then a*f(n) is also Θ(g(n)) ; where a is a constant.
If f(n) is Ω (g(n)) then a*f(n) is also Ω (g(n)) ; where a is a constant.
2. Transitive Properties :
We can say
If f(n) is Θ(g(n)) and g(n) is Θ(h(n)) then f(n) = Θ(h(n)) .
If f(n) is Ω (g(n)) and g(n) is Ω (h(n)) then f(n) = Ω (h(n))
3. Reflexive Properties :
If f(n) is given then f(n) is O(f(n)). Since MAXIMUM VALUE OF f(n) will be f(n) ITSELF !
4. Symmetric Properties :
Efficiency Factors
Space Efficiency
In some cases, the amount of space/memory consumed has to be examined. For example, in dealing
with huge amounts of data or in programming embedded systems, memory consumed must be analyzed.
Space of Instruction: Compiler, compiler settings, and target machine (CPU), all have an impact on space
of instruction.
Data Space: Data size/dynamically allocated memory, static program variables are all factors affecting
data space.
Stack Space at Runtime: Compiler, run-time function calls and recursion, local variables, and
arguments all have an impact on stack space at runtime.
Time Efficiency
Obviously, the faster a program/function completes its objective, the better the algorithm. The actual
running time is determined by a number of factors:
Computer’s Speed: processor (not simply clock speed), I/O, and so on.
Compiler: The compiler, as well as the compiler parameters.
Amount of Data: The amount of data, for example, whether to search a lengthy or a short list.
Actual Data: The actual data, such as whether the name is first or last in a sequential search.
Asymptotic Categorization: This employs basic categories to offer a basic notion of performance, which is
also called an order of magnitude. If the algorithms are similar, the data amount is little, or performance is
essential, then the next method can be explored further.
Estimation of running time depends on two things, i.e., Code Analysis and Code Execution, which
are elaborated below:
Code Analysis: We can accomplish the following things by analyzing the code:
Operation Counts: Choose the most frequently performed operation(s) and count how many times
each one is performed.
Step Counts: Calculate the total number of steps and potentially lines of code that program executes.
Benchmarking: Running the software on a variety of data sets and comparing the results.
Description: A report on the number of hours spent in each routine of a program, which is used to
identify and eliminate the program’s hot spots. This perception is frequently questioned. Although other
description modes give data in units other than time (for example, call counts) and/or at levels of
granularity besides per-routine, the concept is the same.
Worst case
Average case
Best case
The worst-case scenario is evaluated since it provides an upper limit on projected performance.
Furthermore, the average scenario is typically more difficult to establish (it is more data dependent), and it
is frequently the same as the worst case. It may be essential to examine all three situations on some
occasions.
Performance measurement and program evaluation can both help identify areas of programs that need
improvement and determine whether the program is achieving its goals or objectives. They serve different
but complimentary functions:
Performance measurement is an ongoing process that monitors and reports on a program's progress
and accomplishments by using pre-selected performance measures.
Program evaluation, however, uses measurement and analysis to answer specific questions about
how well a program is achieving its outcomes and why.
Program evaluations are individual systematic studies conducted to assess how well a program is working
and why. EPA has used program evaluation to:
Program Evaluation:-
Program evaluations can assess the performance of a program at all stages of a program's
development. The type of program evaluation conducted aligns with the program's maturity (e.g.,
developmental, implementation, or completion) and is driven by the purpose for conducting the evaluation
and the questions that it seeks to answer. The purpose of the program evaluation determines which type of
evaluation is needed.
Design Evaluation:-
Process Evaluation:-
activities and outputs conform to statutory and regulatory requirements, EPA policies, program design or
customer expectations.
Outcome Evaluations:-
Outcome evaluations examine the results of a program (intended or unintended) to determine the
reasons why there are differences between the outcomes and the program's stated goals and objectives
(e.g., why the number and quality of permits issued exceeded or fell short of the established goal?).
Outcome evaluations sometimes examine program processes and activities to better understand how
outcomes are achieved and how quality and productivity could be improved.
Impact Evaluation:-
An impact evaluation is a subset of an outcome evaluation. It assesses the causal links between
program activities and outcomes. This is achieved by comparing the observed outcomes with an estimate
of what would have happened if the program had not existed (e.g., would the water be swimmable if the
program had not been instituted).
Cost-Effectiveness Evaluation:-
Cost-effectiveness evaluations identify program benefits, outputs or outcomes and compare them
with the internal and external costs of the program.
Performance measurement is a way to continuously monitor and report a program's progress and
accomplishments, using pre-selected performance measures. By establishing program measures, offices
can gauge whether their program is meeting their goals and objectives. Performance measures help
programs understand "what" level of performance is achieved.
Measurement is essential to making cost-effective decisions. We strive to meet three key criteria in our
measurement work:
Is it meaningful?
o Measurement should be consistent and comparable to help sustain learning.
Is it credible?
o Effective measurement should withstand reasonable scrutiny.
Is it practical?
o Measurement should be scaled to an agency's needs and budgetary constraints.
A program sets performance measures as a series of goals to meet over time. Measurement data
can be used to identify/flag areas of increasing or decreasing performance that may warrant further
investigation or evaluation. Program evaluations assess whether the program is meeting those
performance measures but also look at why they are or are not meeting them.
For example, imagine you bought a new car that is supposed to get 30 miles per gallon. But say,
you notice that you are only getting 20 miles per gallon. That's a performance measurement. You
looked at whether your car was performing where it should be. So what do you do next? You
would take it to a mechanic. The mechanic's analysis and recommendations would be the program
evaluation because the mechanic would diagnose why the car is not performing as well as it
should.
1) Substitution Method: We make a guess for the solution and then we use mathematical induction to
prove the guess is correct or incorrect.
We need to prove that T(n) <= cnLogn. We can assume that it is true
for values smaller than n.
T(n) = 2T(n/2) + n
<= 2cn/2Log(n/2) + n
= cnLogn - cnLog2 + n
= cnLogn - cn + n
<= cnLogn
Master Method: Master Method is a direct way to get the solution. The master method works only for
following type of recurrences or for recurrences that can be transformed to following type.
Master method is mainly derived from recurrence tree method. If we draw recurrence tree of T(n) =
aT(n/b) + f(n), we can see that the work done at root is f(n) and work done at all leaves is Θ(n c) where c is
Logba. And the height of recurrence tree is Logbn
In recurrence tree method, we calculate total work done. If the work done at leaves is polynomially more,
then leaves are the dominant part, and our result becomes the work done at leaves (Case 1). If work done
at leaves and root is asymptotically same, then our result becomes height multiplied by work done at any
level (Case 2). If work done at root is asymptotically more, then our result becomes work done at root
(Case 3).
Examples of some standard algorithms whose time complexity can be evaluated using Master
Method
Merge Sort: T(n) = 2T(n/2) + Θ(n). It falls in case 2 as c is 1 and Log ba] is also 1. So the solution is
Θ(n Logn)
Binary Search: T(n) = T(n/2) + Θ(1). It also falls in case 2 as c is 0 and Logba is also 0. So the
solution is Θ(Logn)
Notes:
1) It is not necessary that a recurrence of the form T(n) = aT(n/b) + f(n) can be solved using Master
Theorem. The given three cases have some gaps between them. For example, the recurrence T(n) =
2T(n/2) + n/Logn cannot be solved using master method.
Recursion Tree Method is a pictorial representation of an iteration method which is in the form of a
tree where at each level nodes are expanded.
4. It is sometimes difficult to come up with a good guess. In Recursion tree, each root and child represents
the cost of a single subproblem.
5. We sum the costs within each of the levels of the tree to obtain a set of pre-level costs and then sum all
pre-level costs to determine the total cost of all levels of the recursion.
6. A Recursion Tree is best used to generate a good guess, which can be verified by the Substitution
Method.
Example 1
Consider T (n) = 2T + n2
Recurrence Tree Method: In this method, we draw a recurrence tree and calculate the time taken by
every level of tree. Finally, we sum the work done at all levels. To draw the recurrence tree, we start from
the given recurrence and keep drawing till we find a pattern among levels. The pattern is typically a
arithmetic or geometric series.
cn2
/ \
T(n/4) T(n/2)
cn2
/ \
c(n2)/16 c(n2)/4
/ \ / \
T(n/16) T(n/8) T(n/8) T(n/4)
Breaking down further gives us following
cn2
/ \
c(n2)/16 c(n2)/4
/ \ / \
c(n )/256 c(n )/64 c(n2)/64 c(n2)/16
2 2
/ \ / \ / \ / \
Data Structures are the programmatic way of storing data so that data can be used
efficiently. Almost every enterprise application uses various types of data structures in one or
the other way. This tutorial will give you a great understanding on Data Structures needed to
understand the complexity of enterprise level applications and need of algorithms, and data
structures.
As applications are getting complex and data rich, there are three common problems that
applications face now-a-days.
To solve the above-mentioned problems, data structures come to rescue. Data can be
organized in a data structure in such a way that all items may not be required to be searched,
and the required data can be searched almost instantly.
From the data structure point of view, following are some important categories of algorithms −
Binary Search Tree is a node-based binary tree data structure which has the following
properties:
The left subtree of a node contains only nodes with keys lesser than the node’s key.
The right subtree of a node contains only nodes with keys greater than the node’s key.
The left and right subtree each must also be a binary search tree.
The left subtree of a node contains only nodes with keys lesser than the node’s key.
The right subtree of a node contains only nodes with keys greater than the node’s key.
The left and right subtree each must also be a binary search tree.
There must be no duplicate nodes.
The above properties of Binary Search Tree provides an ordering among keys so that the
operations like search, minimum and maximum can be done fast. If there is no ordering, then
we may have to compare every key to search for a given key.
Searching a key : For searching a value, if we had a sorted array we could have performed a
binary search. Let’s say we want to search a number in the array what we do in binary search is
we first define the complete list as our search space, the number can exist only within the search
space. Now we compare the number to be searched or the element to be searched with the mid
element of the search space or the median and if the record being searched is lesser we go
searching in the left half else we go searching in the right half, in case of equality we have
found the element. In binary search we start with ‘n’ elements in search space and then if the
mid element is not the element that we are looking for, we reduce the search space to ‘n/2’ and
we go on reducing the search space till we either find the record that we are looking for or we
get to only one element in search space and be done with this whole reduction.
Search operation in binary search tree will be very similar. Let’s say we want to search
for the number, what we’ll do is we’ll start at the root, and then we will compare the value to be
Searched with the value of the root if it’s equal we are done with the search if it’s lesser
we know that we need to go to the left subtree because in a binary search tree all the elements in
the left subtree are lesser and all the elements in the right subtree are greater. Searching an
element in the binary search tree is basically this traversal in which at each step we will go
either towards left or right and hence in at each step we discard one of the sub-trees. If the tree
is balanced, we call a tree balanced if for all nodes the difference between the heights of left and
right subtrees is not greater than one, we will start with a search space of ‘n’nodes and when we
will discard one of the sub-trees we will discard ‘n/2’ nodes so our search space will be reduced
to ‘n/2’ and then in the next step we will reduce the search space to ‘n/4’ and we will go on
reducing like this till we find the element or till our search space is reduced to only one node.
The search here is also a binary search and that’s why the name binary search tree.
Insertion of a key :-
A new key is always inserted at the leaf. We start searching a key from the root until we hit a
leaf node. Once a leaf node is found, the new node is added as a child of the leaf node.
100 100
/ \ Insert 40 / \
20 500 ---------> 20 500
/ \ / \
10 30 10 30
\
40
Time Complexity: The worst-case time complexity of search and insert operations is O(h)
where h is the height of the Binary Search Tree. In the worst case, we may have to travel from
root to the deepest leaf node. The height of a skewed tree may become n and the time
complexity of search and insert operation may become O(n).
50 50
/ \ delete(20) / \
30 70 ---------> 30 70
/ \ / \ \ / \
20 40 60 80 40 60 80
2) Node to be deleted has only one child: Copy the child to the node and delete the child
50 50
/ \ delete(30) / \
30 70 ---------> 40 70
\ / \ / \
40 60 80 60 80
3) Node to be deleted has two children: Find inorder successor of the node. Copy contents of
the inorder successor to the node and delete the inorder successor. Note that inorder predecessor
can also be used.
50 60
/ \ delete(50) / \
40 70 ---------> 40 70
/ \ \
60 80 80
The important thing to note is, inorder successor is needed only when the right child is not
empty. In this particular case, inorder successor can be obtained by finding the minimum value
in the right child of the node.
Time Complexity: The worst case time complexity of delete operation is O(h) where h is the
height of the Binary Search Tree. In worst case, we may have to travel from the root to the
deepest leaf node. The height of a skewed tree may become n and the time complexity of delete
operation may become O(n)
2.3. Red Black trees: Properties of Red-Black Trees – Rotations – Insertion – Deletion
A red-black tree is a kind of self-balancing binary search tree where each node has an
extra bit, and that bit is often interpreted as the colour (red or black). These colours are used to
ensure that the tree remains balanced during insertions and deletions. Although the balance of
the tree is not perfect, it is good enough to reduce the searching time and maintain it around
O(log n) time, where n is the total number of elements in the tree. This tree was invented in
1972 by Rudolf Bayer.
It must be noted that as each node requires only 1 bit of space to store the colour
information, these types of trees show identical memory footprint to the classic (uncoloured)
binary search tree.
Most of the BST operations (e.g., search, max, min, insert, delete.. etc) take O(h) time
where h is the height of the BST. The cost of these operations may become O(n) for a skewed
Binary tree. If we make sure that the height of the tree remains O(log n) after every insertion
and deletion, then we can guarantee an upper bound of O(log n) for all these operations. The
height of a Red-Black tree is always O(log n) where n is the number of nodes in the tree.
The AVL trees are more balanced compared to Red-Black Trees, but they may cause
more rotations during insertion and deletion. So if your application involves frequent insertions
and deletions, then Red-Black trees should be preferred. And if the insertions and deletions are
less frequent and search is a more frequent operation, then AVL tree should be preferred over
Red-Black Tree.
A simple example to understand balancing is, a chain of 3 nodes is not possible in the
Red-Black tree. We can try any combination of colours and see all of them violate Red-Black
tree property.
1. Black height of the red-black tree is the number of black nodes on a path from the root
node to a leaf node. Leaf nodes are also counted as black nodes. So, a red-black tree of
height h has black height >= h/2.
2. Height of a red-black tree with n nodes is h<= 2 log2(n + 1).
3. All leaves (NIL) are black.
4. The black depth of a node is defined as the number of black nodes from the root to that
node i.e the number of black ancestors.
5. Every red-black tree is a special case of a binary tree.
Black height is the number of black nodes on a path from the root to a leaf. Leaf nodes
are also counted black nodes. From the above properties 3 and 4, we can derive, a Red-Black
Tree of height h has black-height >= h/2.
Number of nodes from a node to its farthest descendant leaf is no more than twice as the
number of nodes to the nearest descendant leaf.
Every Red Black Tree with n nodes has height <= 2Log2(n+1)
This can be proved using the following facts:
1. For a general Binary Tree, let k be the minimum number of nodes on all root to NULL
paths, then n >= 2k – 1 (Ex. If k is 3, then n is at least 7). This expression can also be
written as k <= Log2(n+1).
2. From property 4 of Red-Black trees and above claim, we can say in a Red-Black Tree
with n nodes, there is a root to leaf path with at-most Log2(n+1) black nodes.
3. From property 3 of Red-Black trees, we can claim that the number of black nodes in a
Red-Black tree is at least ⌊ n/2 ⌋ where n is the total number of nodes.
From the above points, we can conclude the fact that Red Black Tree with n nodes has height
<= 2Log2(n+1).
As every red-black tree is a special case of a binary tree so the searching algorithm of a red-
black tree is similar to that of a binary tree.
Algorithm:
Step 2: END
Solution:
In this post, we introduced Red-Black trees and discussed how balance is ensured. The hard part
is to maintain balance when keys are added and removed. We have also seen how to search an
element from the red-black tree. We will soon be discussing insertion and deletion operations in
coming posts on the Red-Black tree.
Left Rotation
Right Rotation
In left rotation, we assume that the right child is not null. Similarly, in the right rotation, we
assume that the left child is not null.
After applying left rotation on the node x, the node y will become the new root of the
subtree and its left child will be x. And the previous left child of y will now become the right
child of x.
Now applying right rotation on the node y of the rotated tree, it will transform back to
the original tree.
So right rotation on the node y will make x the root of the tree, y will become x's right
child. And the previous right child of x will now become the left child of y.
Take a note that rotation doesn't violate the property of binary search trees.
The left grandchild of x (left child of the right child x) will become the right child of it after
rotation. We will do this but before doing this, let's mark the right child of x as y.
LEFT_ROTATION(T, x)
y = x.right
x.right = y.left
The left child of y is going to be the right child of x - x.right = y.left. We also need
to change the parent of y.left to x. We will do this if the left child of y is not NULL.
if y.left != NULL
y.left.parent = x
Then we need to put y to the position of x. We will first change the parent of y to the
parent of x - y.parent = x.parent. After this, we will make the node x the child of y's parent
instead of y. We will do so by checking if y is the right or left child of its parent. We will also
check if y is the root of the tree.
y.parent = x.parent
if x.parent == NULL //x is root
T.root = y
elseif x == x.parent.left // x is left child
x.parent.left = y
else // x is right child
x.parent.right = y
y.left = x
x.parent = y
LEFT_ROTATE(T, x)
y = x.right
x.right = y.left
if y.left != NULL
y.left.parent = x
y.parent = x.parent
T.root = y
x.parent.left = y
x.parent.right = y
y.left = x
x.parent = y
From the above code, you can easily see that rotation is a constant time taking process ( O(1)
).
INSERT(T, n)
y = T.NIL
temp = T.root
y = temp
temp = temp.left
else
temp = temp.right
n.parent = y
if y==T.NIL
T.root = n
y.left = n
else
y.right = n
n.left = T.NIL
n.right = T.NIL
n.color = RED
INSERT_FIXUP(T, n)
Here, we have used T.NIL instead of NULL unlike we do with normal binary search tree.
Also, those T.NIL are the leaves and they all are black, so there won't be a violation of property
3.
In the last few lines, we are making the left and right of the new node T.NIL and also
making it red. At last, we are calling the function to fix the violations of the red-black
properties. Rest of the code is similar to a normal binary search tree.
The property 4 will be violated when the parent of the inserted node is red. So, we will
fix the violations if the parent of the new node is red. At last, we will color the root black and
that will fix the violation of property 2 if it is violated. In the case of violation of property 4
(when the parent of the new node is red), the grandparent will be black.
There can be six cases in the case of violation of the property 4. Let's look at the given
pictures first assuming that the parent of the node z is a left child of its parent which gives us the
first three cases. The other three cases will be symmetric when the node z will be the right child
of its parent.
The first case is when the uncle of z is also red. In this case, we will shift the red color
upward until there is no violation. Otherwise, if it reaches to the root, we can just color it black
without any consequences.
So, the process is to make both the parent and the uncle of z black and its grandparent red. In
this way, the black height of any node won't be affected and we can successfully shift the red
color upward.
However, making the grandparent of z red might cause violation of the property 4 there. So, we
will do the fixup again on that node.
In the second case, the uncle of the node z is black and the node z is the right child.
In the third case, the uncle of the node z is black and the node z is the left child.
We can transform the second case into the third one by performing left rotation on the
parent of the node z. Since both z and its parent are red, so rotation won't affect the black height.
In case 3, we first color the parent of the node z black and its grandparent red and then
do a right rotation on the grandparent of the node z. This fixes the violation of properties
completely. This is shown in the picture given below.
Since we are making the parent of z black in both case 2 and case 3, the loop will stop in these
two cases.
Similarly, there will be three cases when the parent of z will be the right child but those
cases will be symmetric to the above cases only with left and right exchanged.
2.4. B-Trees: Definition of B - trees – Basic operations on B-Trees – Deleting a key from a B-Tree
8. Like other balanced Binary Search Trees, time complexity to search, insert and delete is
O(log n).
9. Insertion of a Node in B-Tree happens only at Leaf Node.
Following is an example of B-Tree of minimum order 5. Note that in practical B-Trees, the
value of the minimum order is much more than 5.
We can see in the above diagram that all the leaf nodes are at the same level and all non-leaf
have no empty sub-tree and have keys one less than the number of their children.
Interesting Facts:
1. The minimum height of the B-Tree that can exist with n number of nodes and m is the
maximum number of children of a node.
2. The maximum height of the B-Tree that can exist with n number of nodes and d is the
minimum number of children that a non-root node.
Traversal in B-Tree: Traversal is also similar to Inorder traversal of Binary Tree. We start
from the leftmost child, recursively print the leftmost child, then repeat the same process for
remaining children and keys. In the end, recursively print the rightmost child.
Search Operation in B-Tree: Search is similar to the search in Binary Search Tree. Let the key
to be searched be k. We start from the root and recursively traverse down. For every visited
non-leaf node, if the node has the key, we simply return the node. Otherwise, we recur down to
the appropriate child (The child which is just before the first greater key) of the node. If we
reach a leaf node and don’t find k in the leaf node, we return NULL.
Logic: Searching a B-Tree is similar to searching a binary tree. The algorithm is similar and
goes with recursion. At each level, the search is optimized as if the key value is not present in
the range of parent then the key is present in another branch. As these values limit the search
they are also known as limiting value or separation value. If we reach a leaf node and don’t find
the desired key then it will display NULL.
Solution:
In this example, we can see that our search was reduced by just limiting the chances
where the key containing the value could be present. Similarly if within the above example
we’ve to look for 180, then the control will stop at step 2 because the program will find that the
key 180 is present within the current node. And similarly, if it’s to seek out 90 then as 90 < 100
so it’ll go to the left subtree automatically and therefore the control flow will go similarly as
shown within the above example.
Insertion
1) Initialize x as root.
2) While x is not leaf, do following
..a) Find the child of x that is going to be traversed next. Let the child be y.
..b) If y is not full, change x to point to y.
..c) If y is full, split it and change x to point to one of the two parts of y. If k is smaller than mid
key in y, then set x as the first part of y. Else second part of y. When we split y, we move a key
from y to its parent x.
3) The loop in step 2 stops when x is leaf. x must have space for 1 extra key as we have been
splitting all nodes in advance. So simply insert k to x.
Let us understand the algorithm with an example tree of minimum degree ‘t’ as 3 and a
sequence of integers 10, 20, 30, 40, 50, 60, 70, 80 and 90 in an initially empty B-Tree.
Initially root is NULL. Let us first insert 10.
Let us now insert 20, 30, 40 and 50. They all will be inserted in root because the maximum
number of keys a node can accommodate is 2*t – 1 which is 5.
Let us now insert 60. Since root node is full, it will first split into two, then 60 will be inserted
into the appropriate child.
Let us now insert 70 and 80. These new keys will be inserted into the appropriate leaf without
any split.
Let us now insert 90. This insertion will cause a split. The middle key will go up to the parent.
Deletion from a B-tree is more complicated than insertion, because we can delete a key from
any node-not just a leaf—and when we delete a key from an internal node, we will have to
rearrange the node’s children.
As in insertion, we must make sure the deletion doesn’t violate the B-tree properties. Just as
we had to ensure that a node didn’t get too big due to insertion, we must ensure that a node
doesn’t get too small during deletion (except that the root is allowed to have fewer than the
minimum number t-1 of keys). Just as a simple insertion algorithm might have to back up if a
node on the path to where the key was to be inserted was full, a simple approach to deletion
might have to back up if a node (other than the root) along the path to where the key is to be
deleted has the minimum number of keys.
The deletion procedure deletes the key k from the subtree rooted at x. This procedure
guarantees that whenever it calls itself recursively on a node x, the number of keys in x is at
least the minimum degree t . Note that this condition requires one more key than the minimum
required by the usual B-tree conditions, so that sometimes a key may have to be moved into a
child node before recursion descends to that child. This strengthened condition allows us to
delete a key from the tree in one downward pass without having to “back up” (with one
exception, which we’ll explain). You should interpret the following specification for deletion
from a B-tree with the understanding that if the root node x ever becomes an internal node
having no keys (this situation can occur in cases 2c and 3b then we delete x, and x’s only child
x.c1 becomes the new root of the tree, decreasing the height of the tree by one and preserving
the property that the root of the tree contains at least one key (unless the tree is empty).
We sketch how deletion works with various cases of deleting keys from a B-tree.
a) If the child y that precedes k in node x has at least t keys, then find the predecessor k0 of k
in the sub-tree rooted at y. Recursively delete k0, and replace k by k0 in x. (We can find k0 and
delete it in a single downward pass.)
b) If y has fewer than t keys, then, symmetrically, examine the child z that follows k in node
x. If z has at least t keys, then find the successor k0 of k in the subtree rooted at z. Recursively
delete k0, and replace k by k0 in x. (We can find k0 and delete it in a single downward pass.)
c) Otherwise, if both y and z have only t-1 keys, merge k and all of z into y, so that x loses
both k and the pointer to z, and y now contains 2t-1 keys. Then free z and recursively delete k
from y.
3. If the key k is not present in internal node x, determine the root x.c(i) of the appropriate
subtree that must contain k, if k is in the tree at all. If x.c(i) has only t-1 keys, execute step 3a or
3b as necessary to guarantee that we descend to a node containing at least t keys. Then finish by
recursing on the appropriate child of x.
a) If x.c(i) has only t-1 keys but has an immediate sibling with at least t keys, give x.c(i) an
extra key by moving a key from x down into x.c(i), moving a key from x.c(i) ’s immediate left
or right sibling up into x, and moving the appropriate child pointer from the sibling into x.c(i).
b) If x.c(i) and both of x.c(i)’s immediate siblings have t-1 keys, merge x.c(i) with one
sibling, which involves moving a key from x down into the new merged node to become the
median key for that node.
Since most of the keys in a B-tree are in the leaves, deletion operations are most often used to
delete keys from leaves. The recursive delete procedure then acts in one downward pass through
the tree, without having to back up. When deleting a key in an internal node, however, the
procedure makes a downward pass through the tree but may have to return to the node from
which the key was deleted to replace the key with its predecessor or successor (cases 2a and
2b).
1) It’s a complete tree (All levels are completely filled except possibly the last level and the last
level has all keys as left as possible). This property of Binary Heap makes them suitable to be
stored in an array.
2) A Binary Heap is either Min Heap or Max Heap. In a Min Binary Heap, the key at root must
be minimum among all keys present in Binary Heap. The same property must be recursively
true for all nodes in Binary Tree. Max Binary Heap is similar to MinHeap.
10 10
/ \ / \
20 100 15 30
/ / \ / \
30 40 50 100 40
Applications of Heaps:
1) Heap Sort: Heap Sort uses Binary Heap to sort an array in O(nLogn) time.
2) Priority Queue: Priority queues can be efficiently implemented using Binary Heap because
it supports insert(), delete() and extractmax(), decreaseKey() operations in O(logn) time.
Binomoial Heap and Fibonacci Heap are variations of Binary Heap. These variations perform
union also efficiently.
3) Graph Algorithms: The priority queues are especially used in Graph Algorithms like
Dijkstra’s Shortest Path and Prim’s Minimum Spanning Tree.
4) Many problems can be efficiently solved using Heaps. See following for example.
a) K’th Largest Element in an array.
b) Sort an almost sorted array/
c) Merge K Sorted Arrays.
4) insert(): Inserting a new key takes O(Logn) time. We add a new key at the end of the
tree. IF new key is greater than its parent, then we don’t need to do anything. Otherwise,
we need to traverse up to fix the violated heap property.
5) delete(): Deleting a key also takes O(Logn) time. We replace the key to be deleted
with minum infinite by calling decreaseKey(). After decreaseKey(), the minus infinite
value must reach root, so we call extractMin() to remove the key.
A max-heap is a complete binary tree in which the value in each internal node is greater
than or equal to the values in the children of that node. Mapping the elements of a heap into an
array is trivial: if a node is stored an index k, then its left child is stored at index 2k+1 and its
right child at index 2k+2.
getMax(): It returns the root element of Max Heap. The Time Complexity of this
operation is O(1).
extractMax(): Removes the maximum element from MaxHeap. The Time Complexity
of this Operation is O(Log n) as this operation needs to maintain the heap property by
calling the heapify() method after removing the root.
insert(): Inserting a new key takes O(Log n) time. We add a new key at the end of the
tree. If the new key is smaller than its parent, then we don’t need to do anything.
Otherwise, we need to traverse up to fix the violated heap property.
Disjoint sets are those sets whose intersection with each other results in a null set. In Set
theory, sometimes we notice that there are no common elements in two sets or we can say that
the intersection of the sets is an empty set or null set. This type of set is called a disjoint set.
For example, if we have X = {a, b, c} and Y = {d, e, f}, then we can say that the given two sets
are disjoint since there are no common elements in these two sets X and Y. In this article, you
will learn what disjoint set is, disjoint set union, Venn diagram, pairwise disjoint set, examples
in detail.
Two sets are said to be disjoint when they have no common element. If a collection has
two or more sets, the condition of disjointness will be the intersection of the entire collection
should be empty.
Yet, a group of sets may have a null intersection without being disjoint. Moreover, while
a group of fewer than two sets is trivially disjoint, since no pairs are there to compare, the
intersection of a group of one set is equal to that set, which may be non-empty. For example, the
three sets {11, 12}, {12, 13}, and {11, 13} have a null intersection but they are not disjoint.
There are no two disjoint sets available in this group. Also, the empty family of sets is pairwise
disjoint.
Two sets A and B are disjoint sets if the intersection of two sets is a null set or an empty set. In
other words, the intersection of a set is empty.
i.e. A ∩ B = ϕ
Properties of Intersection:
Commutative: A ∩ B = B ∩ A
Associative: A ∩ (B ∩ C) = (A ∩ B) ∩ C
A∩∅=∅
A∩B⊆A
A∩A=A
A ⊆ B if and only if A ∩ B = A
A disjoint set union is a binary operation on two sets. The elements of any disjoint union
can be described in terms of ordered pairs as (x, j), where j is the index that represents the origin
of the element x. With the help of this operation, we can join all the different (distinct) elements
of a pair of sets.
A disjoint union may indicate one of two conditions. Most commonly, it may intend the
union of two or more sets that are disjoint. Else if they are disjoint, then their disjoint union may
be produced by adjusting the sets to obtain them disjoint before forming the union of the altered
sets. For example, two sets may be presented as a disjoint set by exchanging each element by an
ordered pair of the element and a binary value symbolising whether it refers to the first or
second set. For groups of more than two sets, one may likewise substitute each element by an
ordered pair of the element and the list of the set that contains it.
Assume that,
X* = { (a, 0), (b, 0), (c, 0), (d, 0) } and Y* = { (e, 1), (f, 1), (g, 1), (h, 1) }
Then,
X U* Y = X* U Y*
Therefore, the disjoint union set is { (a, 0), (b, 0), (c, 0), (d, 0), (e, 1), (f, 1), (g, 1), (h, 1) }
We can proceed with the definition of a disjoint set to any group of sets. A collection of
sets is pairwise disjoint if any two sets in the collection are disjoint. It is also known as mutually
disjoint sets.
Examples:
We know that two sets are disjoint if they don’t have any common elements in the set.
When we take the intersection of two empty sets, the resultant set is also an empty set. One can
easily prove that only the empty sets are disjoint from itself. The following theorem shows that
an empty set is disjoint with itself.
Theorem:
Ø⋂Ø=Ø
Assume that both the sets X and Y are non-empty sets. Thus, X ⋂ Y is also a non-empty
set, the sets are called joint set. In case, if X ⋂ Y results in an empty set, then it is called the
disjoint set.
X ∩ Y = {5}
In case, if
X = {1, 5, 7} and Y = {2, 4, 6}
X∩Y=Ø
Therefore, X and Y are disjoint sets.
Question 1: Show that the given two sets are disjoint sets.
B = {3, 6}
Solution:
That is, A ∩ B = { }
Question 2: Draw a disjoint set Venn diagram that represents the given two sets
Solution:
i.e. A ∩ B = { }
The Venn diagram clearly shows that the given sets are disjoint sets.
Keep visiting BYJU’S – The Learning App to learn more Maths-related concepts and problems.
A fibonacci heap is a data structure that consists of a collection of trees which follow
min heap or max heap property. We have already discussed min heap and max heap property
in the In a fibonacci heap, a node can have more than two children or no children at all. Also, it
has more efficient heap operations than that supported by the binomial and binary heaps.
The fibonacci heap is called a fibonacci heap because the trees are constructed in a way
such that a tree of order n has at least Fn+2 nodes in it, where Fn+2 is the (n + 2)th Fibonacci
number.
Fibonacci Heap
1. It is a set of min heap-ordered trees. (i.e. The parent is always smaller than the children.)
2. A pointer is maintained at the minimum element node.
3. It consists of a set of marked nodes. (Decrease key operation)
4. The trees within a Fibonacci heap are unordered but rooted.
The roots of all the trees are linked together for faster access. The child nodes of a parent node
are connected to each other through a circular doubly linked list as shown below.
There are two main advantages of using a circular doubly linked list.
Algorithm
insert(H, x)
degree[x] = 0
p[x] = NIL
child[x] = NIL
left[x] = x
right[x] = x
mark[x] = FALSE
concatenate the root list containing x with root list H
if min[H] == NIL or key[x] < key[min[H]]
then min[H] = x
n[H] = n[H] + 1
Inserting a node into an already existing heap follows the steps below.
Insertion Example
Find Min
Union
Extract Min
It is the most important operation on a fibonacci heap. In this operation, the node with minimum
value is removed from the heap and the tree is re-adjusted.
Fibonacci Heap
2. Delete the min node, add all its child nodes to the root list and set the min-pointer to the
next root in the root list.
3. The maximum degree in the tree is 3. Create an array of size 4 and map degree of the next roots
with the array.
4. Create an array
5. Here, 23 and 7 have the same degrees, so unite them.
Operations on Heaps
Heap is a very useful data structure that every programmer should know well. The heap data
structure is used in Heap Sort, Priority Queues. The understanding of heaps helps us to know
about memory management. In this blog, we will discuss the various operations of the heap data
structure. We have already discussed what are heaps, its structure, types, and its representation
in the array in the last blog. So let’s get started with the operations on a heap.
Operations on Heaps
Heapify
It is a process to rearrange the elements of the heap in order to maintain the heap property. It is
done when a certain node causes an imbalance in the heap due to some operation on that node.
up_heapify() → It follows the bottom-up approach. In this, we check if the nodes are following
heap property by going in the direction of rootNode and if nodes are not following the heap
property we do certain operations to let the tree follows the heap property.
down_heapify() → It follows the top-down approach. In this, we check if the nodes are
following heap property by going in the direction of the leaf nodes and if nodes are not
following the heap property we do certain operations to let the tree follows the heap property.
Pseudo Code
void down_heapify(int heap[], int parent, int size)
{
largest = parent
leftChild = 2*parent + 1
rightChild = 2*parent + 2
Insertion
9
/ \
5 3
/\
1 4
}
void insert(int heap[],int size,int key)
{
heap.append(key)
up_heapify(heap,size+1,size)
}
Deletion
Now, the last element is placed at some position in heap, it may not follow the property of the
heap, so we need to perform down_heapify() operation in order to maintain heap structure. The
down_heapify() operation does the heapify in the top-bottom approach.
The standard deletion on Heap is to delete the element present at the root node of the heap.
Step 1: Replace the last element with root, and delete it.
4
/ \
6 3
/
1
The maximum element and the minimum element in the max-heap and min-heap is
found at the root node of the heap.
Extract Min-Max
This operation returns and deletes the maximum or minimum element in max-heap and
min-heap respectively. The maximum element is found at the root node.
Bounding the maximum degree: To prove that the amortized time of FIB-HEAP-
EXTRACT-MIN and FIB-HEAP-DELETE is O(lg n), we must show that the upper bound D(n)
on the degree of any node of an n-node Fibonacci heap is O(lg n). The cuts that occur in FIB-
HEAP-DECREASE-KEY, however, may cause trees within the Fibonacci heap to violate the
unordered binomial tree properties. In this section, we shall show that because we cut a node
from its parent as soon as it loses two children, D(n) is O(lg n). In particular, we shall show that
D(n) ≤ ⌊logφn⌋, where .
The key to the analysis is as follows. For each node x within a Fibonacci heap, define
size(x) to be the number of nodes, including x itself, in the subtree rooted at x. (Note that x need
not be in the root list-it can be any node at all.) We shall show that size(x) is exponential in
degree[x]. Bear in mind that degree[x] is always maintained as an accurate count of the degree
of x.
Lemma 20.1
Let x be any node in a Fibonacci heap, and suppose that degree[x] = k. Let y1, y2, . . . , yk denote
the children of x in the order in which they were linked to x, from the earliest to the latest. Then,
degree[y1] ≥ 0 and degree[yi] ≥ i - 2 for i = 2, 3, . . . , k.
For i ≥ 2, we note that when yi was linked to x, all of y1, y2, . . . , yi-1 were children of x, so we
must have had degree[x] = i - 1. Node yi is linked to x only if degree[x] = degree[yi], so we must
have also had degree[yi] = i - 1 at that time. Since then, node yi has lost at most one child, since
it would have been cut from x if it had lost two children. We conclude that degree[yi ] ≥ i - 2.
We finally come to the part of the analysis that explains the name "Fibonacci heaps." Recall
from Standard notations and common functions that for k = 0, 1, 2, . . . , the kth Fibonacci
number is defined by the recurrence
Lemma 20.2
The following lemma and its corollary complete the analysis. They use the in-equality
Fk 2 ≥ φk,
There are two standard ways to represent a graph G = (V, E): as a collection of adjacency lists or
as an adjacency matrix. The adjacency-list representation is usually preferred, because it provides a
compact way to represent sparse graphs--those for which |E| is much less than |V|2. Most of the graph
algorithms presented in this book assume that an input graph is represented in adjacency-list form. An
adjacency-matrix representation may be preferred, however, when the graph is dense--|E| is close to
|V|2 -- or when we need to be able to tell quickly if there is an edge connecting two given vertices. For
example, two of the all-pairs shortest-paths algorithms presented in Chapter 26 assume that their input
graphs are represented by adjacency matrices.
The adjacency-list representation of a graph G = (V, E) consists of an array Adj of |V| lists, one
for each vertex in V. For each u V, the adjacency list Adj[u] contains (pointers to) all the vertices v
such that there is an edge (u,v) E. That is, Adj[u] consists of all the vertices adjacent to u in G. The
vertices in each adjacency list are typically stored in an arbitrary order. Figure 23.1(b) is an adjacency-
list representation of the undirected graph in Figure 23.1(a). Similarly, Figure 23.2(b) is an adjacency-
list representation of the directed graph in Figure 23.2(a).
Figure 23.1 Two representations of an undirected graph. (a) An undirected graph G having five
vertices and seven edges. (b) An adjacency-list representation of G. (c) The adjacency-matrix
representation of G.
Figure 23.2 Two representations of a directed graph. (a) A directed graph G having six vertices
and eight edges. (b) An adjacency-list representation of G. (c) The adjacency-matrix
representation of G.
If G is a directed graph, the sum of the lengths of all the adjacency lists is |E|, since an edge of
the form (u,v) is represented by having v appear in Adj[u]. If G is an undirected graph, the sum of the
lengths of all the adjacency lists is 2|E|, since if (u,v) is an undirected edge, then u appears in v's
adjacency list and vice versa. Whether a graph is directed or not, the adjacency-list representation has
the desirable property that the amount of memory it requires is O(max(V, E)) = O(V + E).
Adjacency lists can readily be adapted to represent weighted graphs, that is, graphs for which
each edge has an associated weight, typically given by a weight function w : E R. For example, let G
= (V, E) be a weighted graph with weight function w. The weight w(u,v) of the edge (u,v) E is simply
stored with vertex v in u's adjacency list. The adjacency-list representation is quite robust in that it can
be modified to support many other graph variants.
For the adjacency-matrix representation of a graph G = (V, E), we assume that the vertices are
numbered 1, 2, . . . , |V| in some arbitrary manner. The adjacency-matrix representation of a graph G
then consists of a |V| |V| matrix A = (aij) such that
Figures 23.1(c) and 23.2(c) are the adjacency matrices of the undirected and directed graphs in
Figures 23.1(a) and 23.2(a), respectively. The adjacency matrix of a graph requires (V2) memory,
independent of the number of edges in the graph.
Observe the symmetry along the main diagonal of the adjacency matrix in Figure 23.1(c). We
define the the transpose of a matrix A = (aij) to be the matrix given by . Since in
an undirected graph, (u,v) and (v,u) represent the same edge, the adjacency matrix A of an undirected
graph is its own transpose: A = AT. In some applications, it pays to store only the entries on and above
the diagonal of the adjacency matrix, thereby cutting the memory needed to store the graph almost in
half.
Breadth-First Traversal (or Search) for a graph is similar to Breadth-First Traversal of a tree (See
method 2 of this post). The only catch here is, unlike trees, graphs may contain cycles, so we may
come to the same node again. To avoid processing a node more than once, we use a boolean visited
array. For simplicity, it is assumed that all vertices are reachable from the starting vertex.
For example, in the following graph, we start traversal from vertex 2. When we come to vertex 0,
we look for all adjacent vertices of it. 2 is also an adjacent vertex of 0. If we don’t mark visited
vertices, then 2 will be processed again and it will become a non-terminating process. A Breadth-First
Traversal of the following graph is 2, 0, 3, 1.
Following are the implementations of simple Breadth-First Traversal from a given source.
The implementation uses an adjacency list representation of graphs. STL‘s list container is used to
store lists of adjacent nodes and the queue of nodes needed for BFS traversal.
Graph::Graph(int V)
{
this->V = V;
adj = new list<int>[V];
}
void Graph::BFS(int s)
{
// Mark all the vertices as not visited
bool *visited = new bool[V];
for(int i = 0; i < V; i++)
visited[i] = false;
// vertices of a vertex
list<int>::iterator i;
while(!queue.empty())
{
// Dequeue a vertex from queue and print it
s = queue.front();
cout << s << " ";
queue.pop_front();
return 0;
}
Output:
Illustration :
Note that the above code traverses only the vertices reachable from a given source vertex. All the
vertices may not be reachable from a given vertex (for example Disconnected graph). To print all the
vertices, we can modify the BFS function to do traversal starting from all nodes one by one (Like the
DFS modified version).
Time Complexity: O(V+E) where V is a number of vertices in the graph and E is a number of edges in
the graph.
Depth First Traversal (or Search) for a graph is similar to Depth First Traversal of a tree. The only
catch here is, unlike trees, graphs may contain cycles, a node may be visited twice. To avoid
processing a node more than once, use a boolean visited array.
Example:
Attention reader! Don’t stop learning now. Get hold of all the important DSA concepts with the
DSA Self Paced Course at a student-friendly price and become industry ready. To complete your
preparation from learning a language to DS Algo and many more, please refer Complete Interview
Preparation Course.
In case you wish to attend live classes with experts, please refer DSA Live Classes for Working
Professionals and Competitive Programming Live for Students.
Input: n = 4, e = 6
0 -> 1, 0 -> 2, 1 -> 2, 2 -> 0, 2 -> 3, 3 -> 3
Output: DFS from vertex 1 : 1 2 0 3
Explanation:
DFS Diagram:
Input: n = 4, e = 6
2 -> 0, 0 -> 2, 1 -> 2, 0 -> 1, 3 -> 3, 1 -> 3
Output: DFS from vertex 2 : 2 0 1 3
Explanation:
DFS Diagram:
Solution:
Approach: Depth-first search is an algorithm for traversing or searching tree or graph data
structures. The algorithm starts at the root node (selecting some arbitrary node as the root
node in the case of a graph) and explores as far as possible along each branch before
backtracking. So the basic idea is to start from the root or any arbitrary node and mark the
node and move to the adjacent unmarked node and continue this loop until there is no
unmarked adjacent node. Then backtrack and check for other unmarked nodes and traverse
them. Finally, print the nodes in the path.
Algorithm:
Create a recursive function that takes the index of the node and a visited array.
Mark the current node as visited and print the node.
Traverse all the adjacent and unmarked nodes and call the recursive function with
the index of the adjacent node.
Implementation:
class Graph
{
public:
map<int, bool> visited;
map<int, list<int>> adj;
void Graph::DFS(int v)
{
// Mark the current node as visited and
// print it
visited[v] = true;
cout << v << " ";
// Driver code
int main()
{
// Create a graph given in the above diagram
Graph g;
g.addEdge(0, 1);
g.addEdge(0, 2);
g.addEdge(1, 2);
g.addEdge(2, 0);
g.addEdge(2, 3);
g.addEdge(3, 3);
return 0;
}
// improved by Vishnudev C
Output:
Complexity Analysis:
Time complexity: O(V + E), where V is the number of vertices and E is the number of edges in the
graph.
Space Complexity: O(V).
Since an extra visited array is needed of size V.
Implementation:
class Graph {
public:
map<int, bool> visited;
map<int, list<int>> adj;
// function to add an edge to graph
void addEdge(int v, int w);
void Graph::DFSUtil(int v)
{
// Mark the current node as visited and print it
visited[v] = true;
cout << v << " ";
// Driver Code
int main()
{
// Create a graph given in the above diagram
Graph g;
g.addEdge(0, 1);
g.addEdge(0, 9);
g.addEdge(1, 2);
g.addEdge(2, 0);
g.addEdge(2, 3);
g.addEdge(9, 3);
return 0;
}
//improved by Vishnudev C
Output:
Complexity Analysis:
Time complexity: O(V + E), where V is the number of vertices and E is the number of edges
in the graph.
Space Complexity :O(V).
Since an extra visited array is needed of size V.
A topological sort on a Graph G(V,E) is a arrangement of its nodes so that all the edges point from
left to right. This can be useful in cases where say there is a ordering of a certain set of events and such
events are repressented by a directed edge from node u to v if u occurs before v. Now if we want to
have an arrangement of node so that they are ordered as written above we use the following algorithm :
Topological Sort(G)
This algo works because a node which is first finished is put into the list and the next finshed node
is put in next. The last node to finish will be on top of the list and this will be the left most event.
A strongly connected component of a digraph is a MAXIMAL set of vertices C such that for every
pair of vertices there is a path from u to v and from v to u.
We can make use of DFS to find the SCCs of a digraph using the following algortihm
SCC(G)
Given an undirected and connected graph , a spanning tree of the graph is a tree that spans (that is,
it includes every vertex of ) and is a subgraph of (every edge in the tree belongs to ).
The cost of the spanning tree is the sum of the weights of all the edges in the tree. There can be
many spanning trees. Minimum spanning tree is the spanning tree where the cost is minimum among
all the spanning trees. There also can be many minimum spanning trees.
Minimum spanning tree has direct application in the design of networks. It is used in algorithms
approximating the travelling salesman problem, multi-terminal minimum cut problem and minimum-
cost weighted perfect matching. Other practical applications are:
1. Cluster Analysis
2. Handwriting recognition
3. Image segmentation
Kruskal’s Algorithm:-
Kruskal’s Algorithm builds the spanning tree by adding edges one by one into a growing spanning
tree. Kruskal's algorithm follows greedy approach as in each iteration it finds an edge which has least
weight and add it to the growing spanning tree.
Algorithm Steps:
This could be done using DFS which starts from the first vertex, then check if the second vertex is
visited or not. But DFS will make time complexity large as it has an order of where is the number of
vertices, is the number of edges. So the best solution is "Disjoint Sets":
Disjoint sets are sets whose intersection is the empty set so it means that they don't have any element in
common.
In Kruskal’s algorithm, at each iteration we will select the edge with the lowest weight. So, we
will start with the lowest weighted edge first i.e., the edges with weight 1. After that we will select the
second lowest weighted edge i.e., edge with weight 2. Notice these two edges are totally disjoint. Now,
the next edge will be the third lowest weighted edge i.e., edge with weight 3, which connects the two
disjoint pieces of the graph. Now, we are not allowed to pick the edge with weight 4, that will create a
cycle and we can’t have any cycles. So we will select the fifth lowest weighted edge i.e., edge with
weight 5. Now the other two edges will create cycles so we will ignore them. In the end, we end up
with a minimum spanning tree with total cost 11 ( = 1 + 2 + 3 + 5).
Implementation:
#include <iostream>
#include <vector>
#include <utility>
#include <algorithm>
void initialize()
{
for(int i = 0;i < MAX;++i)
id[i] = i;
}
int root(int x)
{
while(id[x] != x)
{
id[x] = id[id[x]];
x = id[x];
}
return x;
}
int main()
{
int x, y;
long long weight, cost, minimumCost;
initialize();
cin >> nodes >> edges;
for(int i = 0;i < edges;++i)
{
cin >> x >> y >> weight;
p[i] = make_pair(weight, make_pair(x, y));
}
// Sort the edges in the ascending order
sort(p, p + edges);
minimumCost = kruskal(p);
cout << minimumCost << endl;
return 0;
}
Time Complexity: In Kruskal’s algorithm, most time consuming operation is sorting because the total
complexity of the Disjoint-Set operations will be
Prim’s Algorithm
Prim’s Algorithm also use Greedy approach to find the minimum spanning tree. In Prim’s
Algorithm we grow the spanning tree from a starting position. Unlike an edge in Kruskal's, we add
vertex to the growing spanning tree in Prim's.
Algorithm Steps:
Maintain two disjoint sets of vertices. One containing vertices that are in the growing spanning tree and
other that are not in the growing spanning tree.
Select the cheapest vertex that is connected to the growing spanning tree and is not in the growing
spanning tree and add it into the growing spanning tree. This can be done using Priority Queues. Insert
the vertices, that are connected to growing spanning tree, into the Priority Queue.
Check for cycles. To do that, mark the nodes which have been already selected and insert only those
nodes in the Priority Queue that are not marked.
In Prim’s Algorithm, we will start with an arbitrary node (it doesn’t matter which one) and
mark it. In each iteration we will mark a new vertex that is adjacent to the one that we have already
marked. As a greedy algorithm, Prim’s algorithm will select the cheapest edge and mark the vertex. So
we will simply choose the edge with weight 1. In the next iteration we have three options, edges with
weight 2, 3 and 4. So, we will select the edge with weight 2 and mark the vertex. Now again we have
three options, edges with weight 3, 4 and 5. But we can’t choose edge with weight 3 as it is creating a
cycle. So we will select the edge with weight 4 and we end up with the minimum spanning tree of total
cost 7 ( = 1 + 2 +4).
Implementation:
#include <iostream>
#include <vector>
#include <queue>
#include <functional>
#include <utility>
int main()
{
int nodes, edges, x, y;
long long weight, minimumCost;
cin >> nodes >> edges;
for(int i = 0;i < edges;++i)
{
cin >> x >> y >> weight;
adj[x].push_back(make_pair(weight, y));
adj[y].push_back(make_pair(weight, x));
}
// Selecting 1 as the starting node
minimumCost = prim(1);
cout << minimumCost << endl;
return 0;
}
3.6. The Bellman-Ford algorithm – Single-Source Shortest paths in Directed Acyclic Graphs
Given a graph and a source vertex src in graph, find shortest paths from src to all vertices in the
given graph.
The graph may contain negative weight edges. We have discussed Dijkstra’s algorithm for this
problem. Dijkstra’s algorithm is a Greedy algorithm and time complexity is O(V+E LogV) (with the
use of Fibonacci heap). Dijkstra doesn’t work for Graphs with negative weight cycle, Bellman-Ford
works for such graphs. Bellman-Ford is also simpler than Dijkstra and suites well for distributed
systems. But time complexity of Bellman-Ford is O(VE), which is more than Dijkstra.
Algorithm
Following are the detailed steps.
Input: Graph and a source vertex src
Output: Shortest distance to all vertices from src. If there is a negative weight cycle, then shortest
distances are not calculated, negative weight cycle is reported.
1) This step initializes distances from the source to all vertices as infinite and distance to the source
itself as 0. Create an array dist[] of size |V| with all values as infinite except dist[src] where src is
source vertex.
2) This step calculates shortest distances. Do following |V|-1 times where |V| is the number of vertices
in given graph.
…..a) Do following for each edge u-v
………………If dist[v] > dist[u] + weight of edge uv, then update dist[v]
………………….dist[v] = dist[u] + weight of edge uv
3) This step reports if there is a negative weight cycle in graph. Do following for each edge u-v
……If dist[v] > dist[u] + weight of edge uv, then “Graph contains negative weight cycle” The idea of
step 3 is, step 2 guarantees the shortest distances if the graph doesn’t contain a negative weight cycle.
If we iterate through all edges one more time and get a shorter path for any vertex, then there is a
negative weight cycle.
How does this work? Like other Dynamic Programming Problems, the algorithm calculates shortest
paths in a bottom-up manner. It first calculates the shortest distances which have at-most one edge in
the path. Then, it calculates the shortest paths with at-most 2 edges, and so on. After the i-th iteration
of the outer loop, the shortest paths with at most i edges are calculated. There can be maximum |V| – 1
edges in any simple path, that is why the outer loop runs |v| – 1 times. The idea is, assuming that there
is no negative weight cycle, if we have calculated shortest paths with at most i edges, then an iteration
over all edges guarantees to give shortest path with at-most (i+1) edges.
Example :-
Let us understand the algorithm with following example graph. The images are taken from this source.
Let the given source vertex be 0. Initialize all distances as infinite, except the distance to the source
itself. Total number of vertices in the graph is 5, so all edges must be processed 4 times.
Attention reader! Don’t stop learning now. Get hold of all the important DSA concepts with the DSA
Self Paced Course at a student-friendly price and become industry ready. To complete your
preparation from learning a language to DS Algo and many more, please refer Complete Interview
Preparation Course.
In case you wish to attend live classes with experts, please refer DSA Live Classes for Working
Professionals and Competitive Programming Live for Students.
Let all edges are processed in the following order: (B, E), (D, B), (B, D), (A, B), (A, C), (D, C), (B, C),
(E, D). We get the following distances when all edges are processed the first time. The first row shows
initial distances. The second row shows distances when edges (B, E), (D, B), (B, D) and (A, B) are
processed. The third row shows distances when (A, C) is processed. The fourth row shows when (D,
C), (B, C) and (E, D) are processed.
The first iteration guarantees to give all shortest paths which are at most 1 edge long. We get the
following distances when all edges are processed second time (The last row shows final values).
The second iteration guarantees to give all shortest paths which are at most 2 edges long. The
algorithm processes all edges 2 more times. The distances are minimized after the second iteration, so
Implementation:
printArr(dist, V);
return;
}
graph->edge[2].src = 1;
graph->edge[2].dest = 2;
graph->edge[2].weight = 3;
BellmanFord(graph, 0);
return 0;
}
Output:
Notes
1) Negative weights are found in various applications of graphs. For example, instead of paying cost
for a path, we may get some advantage if we follow the path.
2) Bellman-Ford works better (better than Dijkstra’s) for distributed systems. Unlike Dijkstra’s where
we need to find the minimum value of all vertices, in Bellman-Ford, edges are considered one by one.
3) Bellman-Ford does not work with undirected graph with negative edges as it will declared as
negative cycle.
Exercise
1) The standard Bellman-Ford algorithm reports the shortest path only if there are no negative weight
cycles. Modify it so that it reports minimum distances even if there is a negative weight cycle.
Given a graph and a source vertex in the graph, find the shortest paths from the source to all
vertices in the given graph.
Dijkstra’s algorithm is very similar to Prim’s algorithm for minimum spanning tree. Like Prim’s
MST, we generate a SPT (shortest path tree) with a given source as a root. We maintain two sets, one
set contains vertices included in the shortest-path tree, other set includes vertices not yet included in
the shortest-path tree. At every step of the algorithm, we find a vertex that is in the other set (set of not
yet included) and has a minimum distance from the source.
Below are the detailed steps used in Dijkstra’s algorithm to find the shortest path from a single source
vertex to all other vertices in the given graph.
Algorithm:-
1) Create a set sptSet (shortest path tree set) that keeps track of vertices included in the shortest-path
tree, i.e., whose minimum distance from the source is calculated and finalized. Initially, this set is
empty.
2) Assign a distance value to all vertices in the input graph. Initialize all distance values as INFINITE.
Assign distance value as 0 for the source vertex so that it is picked first.
3) While sptSet doesn’t include all vertices
….a) Pick a vertex u which is not there in sptSet and has a minimum distance value.
….b) Include u to sptSet.
….c) Update distance value of all adjacent vertices of u. To update the distance values, iterate through
all adjacent vertices. For every adjacent vertex v, if the sum of distance value of u (from source) and
weight of edge u-v, is less than the distance value of v, then update the distance value of v.
The set sptSet is initially empty and distances assigned to vertices are {0, INF, INF, INF, INF, INF,
INF, INF} where INF indicates infinite. Now pick the vertex with a minimum distance value. The
vertex 0 is picked, include it in sptSet. So sptSet becomes {0}. After including 0 to sptSet, update
distance values of its adjacent vertices. Adjacent vertices of 0 are 1 and 7. The distance values of 1 and
7 are updated as 4 and 8. The following subgraph shows vertices and their distance values, only the
vertices with finite distance values are shown. The vertices included in SPT are shown in green colour.
Pick the vertex with minimum distance value and not already included in SPT (not in sptSET).
The vertex 1 is picked and added to sptSet. So sptSet now becomes {0, 1}. Update the distance values
of adjacent vertices of 1. The distance value of vertex 2 becomes 12.
Pick the vertex with minimum distance value and not already included in SPT (not in sptSET).
Vertex 7 is picked. So sptSet now becomes {0, 1, 7}. Update the distance values of adjacent vertices of
7. The distance value of vertex 6 and 8 becomes finite (15 and 9 respectively).
Pick the vertex with minimum distance value and not already included in SPT (not in sptSET). Vertex
6 is picked. So sptSet now becomes {0, 1, 7, 6}. Update the distance values of adjacent vertices of 6.
The distance value of vertex 5 and 8 are updated.
We repeat the above steps until sptSet includes all vertices of the given graph. Finally, we get the
following Shortest Path Tree (SPT).
// A utility function to find the vertex with minimum distance value, from
// the set of vertices not yet included in shortest path tree
int minDistance(int dist[], bool sptSet[])
{
return min_index;
}
dijkstra(graph, 0);
return 0;
}
Output:
Dynamic programming helps to solve a complex problem by breaking it down into a collection of
simpler subproblems, solving each of those sub-problems just once, and storing their solutions. A
dynamic programming algorithm will examine the previously solved sub problems and will combine
their solutions to give the best solution for the given problem.
Solve bottom-up, building a table of solved subproblems that are used to solve larger ones.
source shortest distances in O(VE) time using Bellman–Ford Algorithm. For a graph with no negative
weights, we can do better and calculate single source shortest distances in O(E + VLogV) time
using Dijkstra’s algorithm. Can we do even better for Directed Acyclic Graph (DAG)? We can
calculate single source shortest distances in O(V+E) time for DAGs. The idea is to use Topological
Sorting.
We initialize distances to all vertices as infinite and distance to source as 0, then we find a topological
sorting of the graph. Topological Sorting of a graph represents a linear ordering of the graph. Once we
have topological order (or linear representation), we one by one process all vertices in topological
order. For every vertex being processed, we update distances of its adjacent using distance of current
vertex.
1) Initialize dist[] = {INF, INF, ….} and dist[s] = 0 where s is the source vertex.
2) Create a toplogical order of all vertices.
3) Do following for every vertex u in topological order.
………..Do following for every adjacent vertex v of u
………………if (dist[v] > dist[u] + weight(u, v))
………………………dist[v] = dist[u] + weight(u, v)
http://www.stoimen.com/blog/2012/10/28/computer-algorithms-shortest-path-in-a-directed-acyclic-
graph/
1. Topologically sort G into L;
2. Set the distance to the source to 0;
3. Set the distances to all other vertices to infinity;
4. For each vertex u in L
5. - Walk through all neighbors v of u;
6. - If dist(v) > dist(u) + w(u, v)
7. - Set dist(v) <- dist(u) + w(u, v);
// The function to find shortest paths from given vertex. It uses recursive
// topologicalSortUtil() to get topological sorting of given graph.
void Graph::shortestPath(int s)
{
stack<int> Stack;
int dist[V];
// Mark all the vertices as not visited
bool *visited = new bool[V];
for (int i = 0; i < V; i++)
visited[i] = false;
// Call the recursive helper function to store Topological Sort
// starting from all vertices one by one
for (int i = 0; i < V; i++)
if (visited[i] == false)
topologicalSortUtil(i, visited, Stack);
// Initialize distances to all vertices as infinite and distance
// to source as 0
for (int i = 0; i < V; i++)
dist[i] = INF;
dist[s] = 0;
// Process vertices in topological order
while (Stack.empty() == false)
{
// Get the next vertex from topological order
int u = Stack.top();
Stack.pop();
// Update distances of all adjacent vertices
list<AdjListNode>::iterator i;
if (dist[u] != INF)
{
for (i = adj[u].begin(); i != adj[u].end(); ++i)
if (dist[i->getV()] > dist[u] + i->getWeight())
dist[i->getV()] = dist[u] + i->getWeight();
}
}
}
// Push current vertex to stack which stores topological sort
Stack.push(v);
}
Time Complexity: Time complexity of topological sorting is O(V+E). After finding topological order,
the algorithm process all vertices and for every vertex, it runs a loop for all adjacent vertices. Total
adjacent vertices in a graph is O(E). So the inner loop runs O(V+E) times. Therefore, overall time
complexity of this algorithm is O(V+E).
DAG shortest paths : Solve the single-source shortest-path problem in a weighted directed acyclic
graph by 1) doing a topological sort on the vertices by edge so vertices with no incoming edges are
first and vertices with only incoming edges are last, 2) assign an infinite distance to every vertex
(dist(v)=∞) and a zero distance to the source, and 3) for each vertex v in sorted order, for each
outgoing edge e(v,u), if dist(v) + weight(e) < dist(u), set dist(u)=dist(v) + weight(e) and the
predecessor of u to v.
You are going to manage a matrix of shortest paths. Call it SP, and SP[i][j], at the end of the
algorithm, will contain the shortest path from node i to node j. You initialize SP to be the adjacency
matrix for a graph. If there is no edge from i to j, then initialize SP[i][j] to be infinity or an
appropriately high sentinel value. You should also initialize SP[i][i] to be zero.
That's it. This is clearly O(|V|3), which is better than running Dijkstra's shortest path algorithm from
every node, when the graph is dense. That running time would be O(|V|3log(|V|)).
An Example:-
This is the same example as in the Wikipedia page (at least as of March, 2016. If Wikipedia changes,
go ahead and use the Wayback Machine to make it match up). Here's the graph
You'll note first that it has negative edges. That's ok, as long as there are no negative cycles in the
graph (which there aren't). Now, we're going to work through the algorithm, and what I'll do is at each
step, show you SP both in graphical form and as a matrix. I'm going to sentinelize the lack of an edge
with ∞.
1 2 3 4
----------------------
1 | 0 ∞ -2 ∞
2 | 4 0 3 ∞
3 | ∞ ∞ 0 2
4 | ∞ -1 ∞ 0
Step 1: i = 1. We start with i = 1. We next have to iterate through every pair of nodes (x,y), and test to
see if the path from x to y through node i is shorter than SP[x][y]. Obviously, we can ignore the cases
when x=i, y=i or x=y. Here are the values that we test: I've colored the "winners" red in each case:
2 3 4 -2 2 3 2
2 4 4 ∞ ∞ 4 4
3 2 ∞ ∞ ∞ 2 2
3 4 ∞ ∞ ∞ 4 4
4 2 ∞ ∞ ∞ 2 2
4 3 ∞ -2 ∞ 3 3
As you can see, there is one place where SP is improved. I'll update the drawing and the matrix below.
I've colored the changed edges/values in red, and I colored node 1 in green to show that it was the
intermediate node in these paths:
1 2 3 4
----------------------
1 | 0 ∞ -2 ∞
2 | 4 0 2 ∞
3 | ∞ ∞ 0 2
4 | ∞ -1 ∞ 0
Step 2: i = 2. Let's make the same table as before, only now with i = 2:
1 3 ∞ 2 ∞ -2 -2
1 4 ∞ ∞ ∞ ∞ ∞
3 1 ∞ 4 ∞ ∞ ∞
3 4 ∞ ∞ ∞ 2 2
4 1 -1 4 3 ∞ 3
4 3 -1 2 1 ∞ 1
There are two places where SP is improved. I'll update the drawing and the matrix below:
1 2 3 4
----------------------
1 | 0 ∞ -2 ∞
2 | 4 0 2 ∞
3 | ∞ ∞ 0 2
4 | 3 -1 1 0
1 2 -2 ∞ ∞ ∞ -2
1 4 -2 2 0 ∞ ∞
2 1 2 ∞ ∞ 4 ∞
2 4 2 2 4 ∞ 2
4 1 1 ∞ ∞ 3 3
4 2 1 ∞ ∞ -1 1
Once again there are two places where SP is improved. I'll update the drawing and the matrix below:
1 2 3 4
----------------------
1 | 0 ∞ -2 0
2 | 4 0 2 4
3 | ∞ ∞ 0 2
4 | 3 -1 1 0
1 2 0 -1 -1 ∞ -1
1 3 0 1 1 -2 -2
2 1 4 3 7 4 4
2 3 4 1 5 2 2
3 1 2 3 5 ∞ 5
3 2 2 -1 1 ∞ 1
We're done -- the final drawing and matrix are below. As you can see, three values were changed, and
there are no more big values on the graph at all.
1 2 3 4
----------------------
1 | 0 -1 -2 0
2 | 4 0 2 4
3 | 5 1 0 2
4 | 3 -1 1 0
The matrix now has your all-pairs shortest paths. If any of the diagonal entries are negative, then your
graph has negative cycles, and the matrix is invalid.
1. Calculate the transitive closure of a binary relation. A binary relation R over a set of numbers is a
definition such that for each i and j in the set, R(i,j) equals 0 or 1. The transitive closure of R is a new
relation R' such that if R(i,j) = 1, then R'(i,j) also equals 1. Moreover, R'(i,j) is minimally transitive. The
transitive property means that if R'(i,k)=1 and R'(k,j)=1, then R'(i,j)=1. The "minimally transitive" part
means that of all possible definitions of R', we choose the one that has the maximum number of i,j pairs
where R'(i,j)=0. I know that was mathy, but we'll give it a practical instance below.
2. Calculate the maximum flow paths between every pair of nodes in a directed, weighted graph.
You can see where this problem could have practicality in traffic analysis.
Dynamic programming is basically, recursion plus using common sense. What it means
is that recursion allows you to express the value of a function in terms of other values of that
function. Where the common sense tells you that if you implement your function in a way that
the recursive calls are done in advance, and stored for easy access, it will make your program
faster. This is what we call Memoization - it is memorizing the results of some specific states,
which can then be later accessed to solve other sub-problems.
Fibonacci (n) = 1; if n = 0
Fibonacci (n) = 1; if n = 1
Fibonacci (n) = Fibonacci(n-1) + Fibonacci(n-2)
So, the first few numbers in this series will be: 1, 1, 2, 3, 5, 8, 13, 21... and so on!
void fib () {
fibresult[0] = 1;
fibresult[1] = 1;
for (int i = 2; i<n; i++)
fibresult[i] = fibresult[i-1] + fibresult[i-2];
}
1. Optimization problems.
2. Combinatorial problems.
The optimization problems expect you to select a feasible solution, so that the value of the
required function is minimized or maximized. Combinatorial problems expect you to figure out
the number of ways to do something, or the probability of some event happening.
Show that the problem can be broken down into optimal sub-problems.
Recursively define the value of the solution by expressing it in terms of optimal
solutions for smaller sub-problems.
Compute the value of the optimal solution in bottom-up fashion.
Construct an optimal solution from the computed information.
Bottom Up - I'm going to learn programming. Then, I will start practicing. Then, I will
start taking part in contests. Then, I'll practice even more and try to improve. After
working hard like crazy, I'll be an amazing coder.
Top Down - I will be an amazing coder. How? I will work hard like crazy. How? I'll
practice more and try to improve. How? I'll start taking part in contests. Then? I'll
practicing. How? I'm going to learn programming.
Let us say that you are given a number N, you've to find the number of different ways to
write it as the sum of 1, 3 and 4.
1+1+1+1+1
1+4
4+1
1+1+3
1+3+1
3+1+1
Take care of the base cases. DP0 = DP1 = DP2 = 1, and DP3 = 2.
Implementation:
The technique above, takes a bottom up approach and uses memoization to not compute results
that have already been computed.
Matrix-Chain Multiplication:-
Given a sequence of matrices, find the most efficient way to multiply these matrices
together. The problem is not actually to perform the multiplications, but merely to decide in
which order to perform the multiplications. We have many options to multiply a chain of
matrices because matrix multiplication is associative. In other words, no matter how we
parenthesize the product, the result will be the same. For example, if we had four matrices A, B,
C, and D, we would have:
However, the order in which we parenthesize the product affects the number of simple
arithmetic operations needed to compute the product, or the efficiency. For example, suppose A
is a 10 × 30 matrix, B is a 30 × 5 matrix, and C is a 5 × 60 matrix. Then,
There are 4 matrices of dimensions 10x20, 20x30, 30x40 and 40x30. Let the input 4 matrices be
A, B, C and D. The minimum number of multiplications are obtained by putting parenthesis in
following way ((AB)C)D --> 10*20*30 + 10*30*40 + 10*40*30
Dynamic programming posses two important elements which are as given below:
One of the main characteristics is to split the problem into subproblem, as similar as
divide and conquer approach. The overlapping subproblem is found in that problem where
bigger problems share the same smaller problem. However unlike divide and conquer there are
many subproblems in which overlap cannot be treated distinctly or independently. Basically,
there are two ways for handling the overlapping subproblems:
a. Top down approach - It is also termed as memoization technique. In this, the problem
is broken into subproblem and these subproblems are solved and the solutions are
remembered, in case if they need to be solved in future. Which means that the values are
stored in a data structure, which will help us to reach them efficiently when the same
problem will occur during the program execution.
b. Bottom up approach - It is also termed as tabulation technique. In this, all subproblems
are needed to be solved in advance and then used to build up a solution to the larger
problem.
2. Optimal sub structure
It implies that the optimal solution can be obtained from the optimal solution of its
subproblem. So optimal substructure is simply an optimal selection among all the possible
substructures that can help to select the best structure of the same kind to exist.
1. Stages: The given problem can be divided into a number of subproblems which are called
stages. A stage is a small portion of given problem.
2. States:This indicates the subproblem for which the decision has to be taken. The variables
which are used for taking a decision at every stage that is called as a state variable.
3. Decision:At every stage, there can be multiple decisions out of which one of the best decisions
should be taken. The decision taken at each stage should be optimal; this is called as a stage
decision.
4. Optimal policy:It is a rule which determines the decision at each and every stage; a policy is
called an optimal policy if it is globally optimal. This is called as Bellman principle of
optimality.
LCS Problem Statement: Given two sequences, find the length of longest subsequence present
in both of them. A subsequence is a sequence that appears in the same relative order, but not
necessarily contiguous. For example, “abc”, “abg”, “bdf”, “aeg”, ‘”acefg”, .. etc are
subsequences of “abcdefg”.
In order to find out the complexity of brute force approach, we need to first know the
number of possible different subsequences of a string with length n, i.e., find the number of
subsequences with lengths ranging from 1,2,..n-1. Recall from theory of permutation and
combination that number of combinations with 1 element are nC1. Number of combinations with
2 elements are nC2 and so forth and so on. We know that nC0 + nC1 + nC2 + … nCn = 2n. So a
string of length n has 2n-1 different possible subsequences since we do not consider the
subsequence with length 0. This implies that the time complexity of the brute force approach
will be O(n * 2n). Note that it takes O(n) time to check if a subsequence is common to both the
strings. This time complexity can be improved using dynamic programming.
It is a classic computer science problem, the basis of diff (a file comparison program that
outputs the differences between two files), and has applications in bioinformatics.
Examples:-
LCS for input Sequences “ABCDGH” and “AEDFHR” is “ADH” of length 3.
LCS for input Sequences “AGGTAB” and “GXTXAYB” is “GTAB” of length 4.
Greedy Algorithms:-
Greedy is an algorithmic paradigm that builds up a solution piece by piece, always choosing
the next piece that offers the most obvious and immediate benefit. So the problems where
choosing locally optimal also leads to global solution are best fit for Greedy.
For example consider the Fractional Knapsack Problem. The local optimal strategy is to
choose the item that has maximum value vs weight ratio. This strategy also leads to global
optimal solution because we allowed to take fractions of an item.
Optimal Substructure:
A thief has a knapsack that holds at most W pounds. Item i : ( vi, wi ) ( v = value, w =
weight ) thief must choose items to maximize the value stolen and still fit into the knapsack.
Each item must be taken or left ( 0 - 1 ).
Both the 0 - 1 and fractional problems have the optimal substructure property:
Fractional: vi / wi is the value per pound. Clearly you take as much of the item with the greatest
value per pound. This continues until you fill the knapsack. Optimal (Greedy) algorithm takes O
( n lg n ), as we must sort on vi / wi = di.
w1 = 10 v1 = 60 d1.= 6
w2 = 20 v2 = 100 d2.= 5
w3 = 30 v3 = 120 d3 = 4
were d is the value density
Greedy approach: Take all of 1, and all of 2: v1+ v2 = 160, optimal solution is to take all of 2
and 3: v2 + v3= 220, other solution is to take all of 1 and 3 v1+ v3 = 180. All below 50 lbs.
When solving the 0 - 1 knapsack problem, empty space lowers the effective d of the load. Thus
each time an item is chosen for inclusion we must consider both
i included
i excluded
These are clearly overlapping sub-problems for different i's and so best solved by DP.
Huffman coding is a lossless data compression algorithm. The idea is to assign variable-
length codes to input characters, lengths of the assigned codes are based on the frequencies of
corresponding characters. The most frequent character gets the smallest code and the least
frequent character gets the largest code.
The variable-length codes assigned to input characters are Prefix Codes, means the
codes (bit sequences) are assigned in such a way that the code assigned to one character is not
the prefix of code assigned to any other character. This is how Huffman Coding makes sure that
there is no ambiguity when decoding the generated bitstream.
Let us understand prefix codes with a counter example. Let there be four characters a, b, c and
d, and their corresponding variable length codes be 00, 01, 0 and 1. This coding leads to
ambiguity because code assigned to c is the prefix of codes assigned to a and b. If the
compressed bit stream is 0001, the de-compressed output may be “cccd” or “ccb” or “acd” or
“ab”.
There are mainly two major parts in Huffman Coding
1. Create a leaf node for each unique character and build a min heap of all leaf nodes (Min
Heap is used as a priority queue. The value of frequency field is used to compare two
nodes in min heap. Initially, the least frequent character is at root)
2. Extract two nodes with the minimum frequency from the min heap.
3. Create a new internal node with a frequency equal to the sum of the two nodes
frequencies. Make the first extracted node as its left child and the other extracted node as
its right child. Add this node to the min heap.
4. Repeat steps#2 and #3 until the heap contains only one node. The remaining node is the
root node and the tree is complete.
Let us understand the algorithm with an example:
character Frequency
a 5
b 9
c 12
d 13
e 16
f 45
Step 1. Build a min heap that contains 6 nodes where each node represents root of a tree with
single node.
Step 2 Extract two minimum frequency nodes from min heap. Add a new internal node with
frequency 5 + 9 = 14.
Now min heap contains 5 nodes where 4 nodes are roots of trees with single element each, and
one heap node is root of tree with 3 elements
character Frequency
c 12
d 13
Internal Node 14
e 16
f 45
Step 3: Extract two minimum frequency nodes from heap. Add a new internal node with
frequency 12 + 13 = 25
Now min heap contains 4 nodes where 2 nodes are roots of trees with single element each, and
two heap nodes are root of tree with more than one nodes
character Frequency
Internal Node 14
e 16
Internal Node 25
f 45
Step 4: Extract two minimum frequency nodes. Add a new internal node with frequency 14 +
16 = 30
character Frequency
Internal Node 25
Internal Node 30
f 45
Step 5: Extract two minimum frequency nodes. Add a new internal node with frequency 25 +
30 = 55
character Frequency
f 45
Internal Node 55
Step 6: Extract two minimum frequency nodes. Add a new internal node with frequency 45 +
55 = 100
character Frequency
Internal Node 100
Since the heap contains only one node, the algorithm stops here.
Steps to print codes from Huffman Tree: Traverse the tree formed starting from the root.
Maintain an auxiliary array. While moving to the left child, write 0 to the array. While moving
to the right child, write 1 to the array. Print the array when a leaf node is encountered.
character code-word
f 0
c 100
d 101
a 1100
b 1101
e 111
Status of NP Complete problems is another failure story, NP complete problems are problems
whose status is unknown. No polynomial time algorithm has yet been discovered for any NP complete
problem, nor has anybody yet been able to prove that no polynomial-time algorithm exists for any of
them. The interesting part is, if any one of the NP complete problems can be solved in polynomial
time, then all of them can be solved.
P is a set of problems that can be solved by a deterministic Turing machine in Polynomial time.
NP-complete problems are the hardest problems in the NP set. A decision problem L is NP-complete
if:
1) L is in NP (Any given solution for NP-complete problems can be verified quickly, but there is no
efficient known solution).
2) Every problem in NP is reducible to L in polynomial time (Reduction is defined below).
A problem is NP-Hard if it follows property 2 mentioned above, doesn’t need to follow property
1. Therefore, the NP-Complete set is also a subset of the NP-Hard set.
What is Reduction?
Let L1 and L2 be two decision problems. Suppose algorithm A2 solves L2. That is, if y is an input
for L2 then algorithm A2 will answer Yes or No depending upon whether y belongs to L2 or not.
The idea is to find a transformation from L1 to L2 so that algorithm A2 can be part of an algorithm A1
to solve L1.
Learning reduction, in general, is very important. For example, if we have library functions to solve
certain problems and if we can reduce a new problem to one of the solved problems, we save a lot of
time. Consider the example of a problem where we have to find the minimum product path in a given
directed graph where the product of path is the multiplication of weights of edges along the path. If we
have code for Dijkstra’s algorithm to find the shortest path, we can take the log of all weights and use
Dijkstra’s algorithm to find the minimum product path rather than writing a fresh code for this new
problem.
From the definition of NP-complete, it appears impossible to prove that a problem L is NP-
Complete. By definition, it requires us to that show every problem in NP in polynomial time reducible
to L. Fortunately, there is an alternate way to prove it. The idea is to take a known NP-Complete
problem and reduce it to L. If polynomial time reduction is possible, we can prove that L is NP-
Complete by transitivity of reduction (If a NP-Complete problem is reducible to L in polynomial time,
then all problems are reducible to L in polynomial time).
It is always useful to know about NP-Completeness even for engineers. Suppose you are asked to
write an efficient algorithm to solve an extremely important problem for your company. After a lot of
thinking, you can only come up exponential time approach which is impractical. If you don’t know
about NP-Completeness, you can only say that I could not come with an efficient algorithm. If you
know about NP-Completeness and prove that the problem is NP-complete, you can proudly say that
the polynomial time solution is unlikely to exist.
5 . 2 . Polynomial-Time Verification
Before talking about the class of NP-complete problems, it is essential to introduce the notion of a
verification algorithm. Many problems are hard to solve, but they have the property that it easy to
authenticate the solution if one is provided.
Consider the Hamiltonian cycle problem. Given an undirected graph G, does G have a cycle that
visits each vertex exactly once? There is no known polynomial time algorithm for this dispute.
Note: - It means you can't build a Hamiltonian cycle in a graph with a polynomial time even if
there is no specific path is given for the Hamiltonian cycle with the particular vertex, yet you
can't verify the Hamiltonian cycle within the polynomial time
Let us understand that a graph did have a Hamiltonian cycle. It would be easy for someone to
convince of this. They would similarly say: "the period is hv3, v7, v1....v13i.
We could then inspect the graph and check that this is indeed a legal cycle and that it visits all of
the vertices of the graph exactly once. Thus, even though we know of no efficient way to solve the
Hamiltonian cycle problem, there is a beneficial way to verify that a given cycle is indeed a
Hamiltonian cycle.
Definition of Certificate: - A piece of information which contains in the given path of a vertex is
known as certificate
1. P contains in NP
2. P=NP
1. Observe that P contains in NP. In other words, if we can solve a problem in polynomial time,
we can indeed verify the solution in polynomial time. More formally, we do not need to see a
certificate (there is no need to specify the vertex/intermediate of the specific path) to solve the
problem; we can explain it in polynomial time anyway.
2. However, it is not known whether P = NP. It seems you can verify and produce an output of the
set of decision-based problems in NP classes in a polynomial time which is impossible because
according to the definition of NP classes you can verify the solution within the polynomial
time. So this relation can never be held.
Reductions:
The class NP-complete (NPC) problems consist of a set of decision problems (a subset of class
NP) that no one knows how to solve efficiently. But if there were a polynomial solution for even a
single NP-complete problem, then every problem in NPC will be solvable in polynomial time. For this,
we need the concept of reductions.
Suppose there are two problems, A and B. You know that it is impossible to solve problem A in
polynomial time. You want to prove that B cannot be explained in polynomial time. We want to show
that (A ∉ P) => (B ∉ P)
3-color: Given a graph G, can each of its vertices be labeled with one of 3 different colors such that
two adjacent vertices do not have the same label (color).
Coloring arises in various partitioning issues where there is a constraint that two objects cannot
be assigned to the same set of partitions. The phrase "coloring" comes from the original application
which was in map drawing. Two countries that contribute a common border should be colored with
different colors.
It is well known that planar graphs can be colored (maps) with four colors. There exists a polynomial
time algorithm for this. But deciding whether this can be done with 3 colors is hard, and there is no
polynomial time algorithm for it.
NP- Completeness:-
Definition: L is NP-complete if
1. L ϵ NP and
2. L' ≤ p L for some known NP-complete problem L.' Given this formal definition, the complexity
classes are:
NP: is the set of decision problems that can be verified in polynomial time.
NP-Hard: L is NP-hard if for all L' ϵ NP, L' ≤p L. Thus if we can solve L in polynomial time, we can
solve all NP problems in polynomial time.
NP-Complete L is NP-complete if
1. L ϵ NP and
2. L is NP-hard
If any NP-complete problem is solvable in polynomial time, then every NP-Complete problem is also
solvable in polynomial time. Conversely, if we can prove that any NP-Complete problem cannot be
solved in polynomial time, every NP-Complete problem cannot be solvable in polynomial time.
To start the process of being able to prove problems are NP-complete, we need to prove just one
problem H is NP-complete. After that, to show that any problem X is NP-hard, we just need to reduce
H to X. When doing NP-completeness proofs, it is very important not to get this reduction backwards!
If we reduce candidate problem X to known hard problem H, this means that we use H as a step to
solving X. All that means is that we have found a (known) hard way to solve X. However, when we
reduce known hard problem H to candidate problem X, that means we are using X as a step to solve H.
And if we know that H is hard, that means X must also be hard (because if X were not hard, then
neither would H be hard).
So a crucial first step to getting this whole theory off the ground is finding one problem that is
NP-hard. The first proof that a problem is NP-hard (and because it is in NP, therefore NP-complete)
was done by Stephen Cook. For this feat, Cook won the first Turing award, which is the closest
Computer Science equivalent to the Nobel Prize. The “grand-daddy” NP-complete problem that Cook
used is called SATISFIABILITY (or SAT for short).
A Boolean expression is comprised of Boolean variables combined using the operators AND (⋅),
OR (+), and NOT (to negate Boolean variable x we write x¯ ¯ ¯ ). A literal is a Boolean variable or its
negation. A clause is one or more literals OR’ed together. Let E be a Boolean expression over variables
x1,x2,...,xn. The we define Conjunctive Normal Form (CNF) to be a Boolean expression written as a
series of clauses that are AND’ed together. For example,
E=(x5+x7+x8¯ ¯ ¯ ¯ ¯ +x10)⋅(x2¯ ¯ ¯ ¯ ¯ +x3)⋅(x1+x3¯ ¯ ¯ ¯ ¯ +x6) is in CNF, and has three clauses. Now we can
define the problem SAT.
Output: YES if there is an assignment to the variables that makes E true, NO otherwise. Cook proved
that SAT is NP-hard. Explaining Cook’s proof is beyond the scope of this course. But we can briefly
summarize it as follows. Any decision problem Fcan be recast as some language acceptance problem
L:F(I)=YES⇔L(I′)=ACCEPT.
That is, if a decision problem F yields YES on input I, then there is a language L containing string
I′ where I′ is some suitable transformation of input I. Conversely, if F would give answer NO for input
I, then I ‘s transformed version I′ is not in the language L.Turing machines are a simple model of
computation for writing programs that are language acceptors. There is a “universal” Turing machine
that can take as input a description for a Turing machine, and an input string, and return the execution
of that machine on that string. This Turing machine in turn can be cast as a Boolean expression such
that the expression is satisfiable if and only if the Turing machine yields ACCEPT for that string. Cook
used Turing machines in his proof because they are simple enough that he could develop this
transformation of Turing machines to Boolean expressions, but rich enough to be able to compute any
function that a regular computer can compute. The significance of this transformation is that any
decision problem that is performable by the Turing machine is transformable to SAT. Thus, SAT is
NP-hard.
To show that a decision problem X is NP-complete, we prove that X is in NP (normally easy, and
normally done by giving a suitable polynomial-time, non-deterministic algorithm) and then prove that
X is NP-hard. To prove that X is NP-hard, we choose a known NP-complete problem, say A. We
describe a polynomial-time transformation that takes an arbitrary instance I of A to an instance I′ of X.
We then describe a polynomial-time transformation from SLN′ to SLN such that SLN is the solution for
I. The following modules show a number of known NP-complete problems, and also some proofs that
they are NP-complete. The various proofs will link the problems together as shown here:
Figure 28.12.1: We will use this sequence of reductions for the NP Complete Proof
5. 5. NP-Complete Problems.
NP-complete problem, any of a class of computational problems for which no efficient solution
algorithm has been found. Many significant computer-science problems belong to this class—e.g., the
traveling salesman problem, satisfiability problems, and graph-covering problems.
So-called easy, or tractable, problems can be solved by computer algorithms that run in
polynomial time; i.e., for a problem of size n, the time or number of steps needed to find the solution is
a polynomial function of n. Algorithms for solving hard, or intractable, problems, on the other hand,
require times that are exponential functions of the problem size n. Polynomial-time algorithms are
considered to be efficient, while exponential-time algorithms are considered inefficient, because the
execution times of the latter grow much more rapidly as the problem size increases.
A problem is called NP (nondeterministic polynomial) if its solution can be guessed and verified
in polynomial time; nondeterministic means that no particular rule is followed to make the guess. If a
problem is NP and all other NP problems are polynomial-time reducible to it, the problem is NP-
complete. Thus, finding an efficient algorithm for any NP-complete problem implies that an efficient
algorithm can be found for all such problems, since any problem belonging to this class can be recast
into any other member of the class. It is not known whether any polynomial-time algorithms will ever
be found for NP-complete problems, and determining whether these problems are tractable or
intractable remains one of the most important questions in theoretical computer science. When an NP-
complete problem must be solved, one approach is to use a polynomial algorithm to approximate the
solution; the answer thus obtained will not necessarily be optimal but will be reasonably close.
NP-Complete Problems
Following are some NP-Complete problems, for which no polynomial time algorithm is known.
NP-Hard Problems
TSP is NP-Complete:-
The traveling salesman problem consists of a salesman and a set of cities. The salesman has to
visit each one of the cities starting from a certain one and returning to the same city. The challenge of
the problem is that the traveling salesman wants to minimize the total length of the trip
Proof:-
To prove TSP is NP-Complete, first we have to prove that TSP belongs to NP. In TSP, we find
a tour and check that the tour contains each vertex once. Then the total cost of the edges of the tour is
calculated. Finally, we check if the cost is minimum. This can be completed in polynomial time. Thus
TSP belongs to NP.
Secondly, we have to prove that TSP is NP-hard. To prove this, one way is to show that Hamiltonian
cycle ≤p TSP (as we know that the Hamiltonian cycle problem is NPcomplete).
Hence, an instance of TSP is constructed. We create the complete graph G' = (V, E'), where
E′={(i,j):i,j∈Vandi≠j
t(i,j)={01if(i,j)∈Eotherwise
Now, suppose that a Hamiltonian cycle h exists in G. It is clear that the cost of each edge in h is 0
in G as each edge belongs to E. Therefore, h has a cost of 0 in G'. Thus, if graph G has a Hamiltonian
'
Conversely, we assume that G' has a tour h' of cost at most 0. The cost of edges in E' are 0 and 1
by definition. Hence, each edge must have a cost of 0 as the cost of h' is 0. We therefore conclude that
h' contains only edges in E.
PREPARED BY
Mr.R.KARTHIKEYAN ASST.PROF/MCA