Algorithms

Algorithms

Reference Books:
1. CLR Introduction to Algorithms - 3rd Edition, By Thomas H. Cormen

This PDF contains the notes from the standards books and are only meant for
GATE CSE aspirants.

Notes Compiled By-

Manu Thakur
Mtech CSE, IIT Delhi
[email protected]
https://www.facebook.com/Worstguymanu

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Algorithms
An algorithm is any well-defined computational procedure that takes some value, or set of values, as
input and produces some value, or set of values, as output. An algorithm is thus a sequence of
computational steps that transform the input into the output.

A data structure is a way to store and organize data in order to facilitate access and modifications.
No single data structure works well for all purposes, and so it is important to know the strengths
And limitations of several of them.

The running-time of an algorithm on a particular input is the number of primitive operations or

steps executed.

Worst Case T.C the worst-case running time of an algorithm gives us an upper bound on the running
time for any input. Knowing it provides a guarantee that the algorithm will never take any longer.

Advantage of Sorting:-
1. Unsorted array search = O(n)
2. Sorted array search = O(logn)
3. Finding Median, sorted array = O(1)
4. Finding Median, unsorted array = O(n)
5. Building a frequency table
6. Checking duplicates

Insertion Sort

The numbers that we wish to sort are also known as the keys.

Insertion sort works the way we sort a hand of playing cards. We start with an empty left hand and
the cards face down on the table. We then remove one card at a time from the table and insert it
into the correct position in the left hand. To find the correct position for a card, we compare it with

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

each of the cards already in the hand, from right to left. At all times, the cards held in the left hand
are sorted, and these cards were originally the top cards of the pile on the table.

1. Array starting with index 1.

2. Starting from index 2 upto length of array n
3. A[j] will be copied into key.
4. From i=0 to j-1 array is already sorted.
5. While i > 0 && key < A[i]
6. Shifting elements to right A[i+1] = A[i]
7. Decreasing index i.
8. Storing key at correct position.

Loop Invariant:

Initialization: It is true prior to the first iteration of the loop.

Maintenance: If it is true before an iteration of the loop, it remains true before the next iteration.

Termination: When the loop terminates, the invariant gives us a useful property that helps show
that the algorithm is correct.

Note:-
1. Insertion sort, takes time roughly equal to c1*n^2 to sort n items, where c1 is a constant
that does not depend on n.
2. Merge sort, takes time roughly equal to c2*logn (base2), and c2 is another constant that
also does not depend on n.
3. Insertion sort can take different amounts of time to sort two input sequences of the same
size depending on how nearly sorted they already are.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Analysis of insertion sort:

Insertion sort typically has a smaller constant factor than merge sort, so that c1 < c2 Hence,
Insertion sort is better than merge sort for small input.

Worst-case and average-case analysis

The running time of the algorithm is the sum of running times for each statement executed;

The best case occurs if the array is already sorted. Line 5 will execute (n-1) times and while loop will
0 times. Best T.C will be O(n)

The worst case occurs if the array is sorted in reverse order. There will be n(n-1)/2 comparisons and
n(n-1)/2 shifts. Overall worst case T.C will be O(n^2).

Note:-
1. If we replace linear search with Binary Search in sorted part, Insertion sort will still take
O(n^2) T.C due to n(n-1)/2 shifts.
2. If we use Linked List in place of array, Insertion Sort will still take O(n^2) because of n(n-1)/2
comparisons.

Properties:-
1. Insertion sort is an efficient algorithm to sort small number of elements
2. The algorithm sorts the input numbers in place. Input array A contains the sorted output
sequence when the INSERTION-SORT is finished.
3. Insertion sort is stable sorting. A sorting algorithm is said to be stable if two objects with
equal keys appear in the same order in sorted output as they appear in the input unsorted
array.
4. Best case O(n) when array is already sorted
5. Worst case O(n^2) when array is sorted in reverse order.
6. Insertion sort is an Adaptive sorting.

Insertion Sort (Recursive Solution)

Sorted part | Unsorted Part

a. Assume A[0…….. i-1] is already sorted

b. A[I ….. n-1] yet to be sorted
c. Insert A[i] into A[0…. I-1]
d. Recursively sort A[i+1 …… n-1]
e. Base case = i=n-1

Recurrence relation
Since it takes theta(n) time in worst case to insert A[n] in the sorted array A[I …… n-1] we get the
recurrence :

T(n) = T(n-1) + n if n>1 and O(1) if n=1

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Inversions:-

Let A[1..n] be an array of n distinct numbers. If i<j and A[i]>A[j], then the pair (i,j) is called an
inversion of A.

Suppose A={2,3,8,6,1}, there are 5 inversions ⟨2,1⟩ , ⟨3,1⟩, ⟨8,6⟩, ⟨8,1⟩ and ⟨6,1⟩. If array is sorted in
reverse order then there will be maximum number of inversions (n-1) + n(-2) + …. 1 = n(n-1)/2

Average number of inversions will be (0 + n(n-1)/2)/2 = n(n-1)/4

Relationship with insertion sort:

Insertion sort performs the body of the inner loop once for each inversion. Thus the running time of
insertion sort is Theeta(n + d) where d is the number of inversions.
a. In best case there will be no inversion.
b. In worst case, there will be n(n-1)/2 inversions.

Time Complexity
1. Merge procedure can be modified to count number of inversions in Theeta(nlogn) time.
2. For each element, count number of elements which are on right side of it and are smaller
than it. It requires O(n^2) time.

Note: - among O(n^2) sorts, Insertion sort is usually better than selection sort and both are better
than Bubble Sort.

Selection Sort:-

The selection sort algorithm sorts an array by repeatedly finding the minimum element (considering
ascending order) from unsorted part and putting it at the beginning. The algorithm maintains two
subarrays in a given array.
1. The subarray which is already sorted.
2. Remaining subarray which is unsorted.

In every iteration of selection sort, the minimum element from the unsorted subarray is picked and
moved to the sorted subarray.

1. For loop will run for n-1 times, as left nth element will be at its correct position.
2. Initially, we have considered first element as smallest element

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

3. Inner for loop will start from index 2 and go up to n

4. If any element is found smaller than our minimum element
5. Set min = index of smaller element
6. At the end swap them

T(n) = (n-1) + (n-2) + …………………………. + 2 + 1 = n(n-1)/2 = O(n^2)

Properties:-
1. There is no best case. Best, worst, and average all are O(n^2)
2. Selection sort is not an Adaptive sorting algorithm.
3. Selection sort is in-place algorithm.
4. Selection Sort is not stable sorting. Given [2, 2, 1], the '2' will be replaced with 1.

Adaptive Sorting Algorithm:

A sorting algorithm falls into the adaptive sort family if it takes advantage of existing order in its
input. It benefits from the pre-sortedness in the input sequence.
Example- Insertion sort.

Note: - we can have adaptive sorting versions of Heap Sort and Merge Sort.

Selection Sort Example

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Selection Sort Recursive Solution:-

a. To sort A[i….. n-1] find minimum value in array and move to A[i]
b. Apply selection sort on A[i+1, n-1]
c. Base Case: do nothing if i=n-1, only on element is left in array

SelectionSort( A, i, n) // Array Name, starting index, total size

1. If(i>= n-1)
2. Return;
3. Lowest_key = i;
4. For(j=i+1 to n-1)
5. If (A[i] > A[j]) then
6. Lowest_key=j;
7. Swap(A[Lowest_key], A[i])
8. SelectionSort(A, i+1, n);

Analysis: n steps to find minimum and move to first position

T(n) = t(n-1) + n
T(n) = T(n-2) + (n-1) + n
.
.
.
= 1 + 2 + ….. + n = n(n+1)/2 = O(n^2)

Bubble Sort:-

A simple sorting algorithm compares each pair of adjacent items and swaps them if they are in the
wrong order. The pass through the list is repeated until no swaps are needed, which indicates that
the list is sorted. It can be practical if the input is usually in sorted order but may occasionally have
some out-of-order elements nearly in position.

1. For loop will execute n-1 times from 1 to n-1

2. It will start from RHS upto i+1
3. If RHS element is less than LHS element
4. Will bubble up smallest element in unsorted part to the front.

Note: - If no swap happens in any pass we can immediately stop. We can use a flag to check this
property. If Array is already sorted we can stop after 1 iteration only i.e O(n)

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Properties:-
1. Worst case T.C is O(n^2).
2. Best Case when array is already sorted O(n).
3. Bubble sort is in-place sorting.
4. Bubble sort is a stable sort.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

The divide-and-conquer approach

Recursive algorithms typically follow a divide-and-conquer approach: they break the problem into
several sub-problems that are similar to the original problem but smaller in size, solve the sub-
problems recursively, and then combine these solutions to create a solution to the original problem.

Divide the problem into a number of subproblems that are smaller instances of the same problem.

Conquer the sub-problems by solving them recursively. If the subproblem sizes are small enough,
however, just solve the subproblems in a straightforward manner.

Combine the solutions to the subproblems into the solution for the original problem.

When the subproblems are large enough to solve recursively, we call that the recursive case. Once
the subproblems become small enough that we no longer recurse, we have gotten down to the base
Case.

Recurrences
A recurrence is an equation or inequality that describes a function in terms of its
Value on smaller inputs.

Master Method:-
The master method provides bounds for recurrences of the form:

T(n) = aT(n/b) + f(n)

Where a>=1 and b>1, and f(n) is a given function.

A recurrence of the form in above equation characterizes a divide and conquer algorithm that
creates a subproblems, each of which is 1/b the size of the original problem, and in which the divide
and combine steps together take f(n) time.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Extended Master Theorem

Occasionally, we shall see recurrences that are not equalities but rather inequalities,

Strassen’s algorithm for matrix multiplication

The SQUARE-MATRIX-MULTIPLY procedure takes Θ(n^3) time. There are three nested for loops.

A simple divide-and-conquer algorithm:

When we use a divide-and-conquer algorithm to compute the matrix product C = A*B, we assume
that n is an exact power of 2 in each of the n x n matrices. We make this assumption because in each
divide step, we will divide n x n matrices into four n/2 x n/2 matrices, and by assuming that n is an
Exact power of 2, we are guaranteed that as long as n >=2, the dimension n/2 is an integer.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Each of these four equations specifies two multiplications of n/2 x n/2 matrices and the addition of
their n/2 x n/2 products.

Let T(n) be the time to multiply two nxn matrices using this procedure. In the base case, when n=1,
we perform just the one scalar multiplication in. T (1) = Θ (1)

The recurrence for the running time of SQUARE-MATRIX-MULTIPLY-RECURSIVE is:

T(n) = Θ(n^3) Thus, this simple divide-and-conquer approach is no faster than the straightforward
SQUARE-MATRIX-MULTIPLY procedure.

Strassen’s method

Instead of performing eight recursive multiplications of n/2xn/2 matrices, it performs only seven.
The cost of eliminating one matrix multiplication will be several new additions of n/2xn/2 matrices,
but still only a constant number of additions.

Thus, this solution has time complexity of T(n) = Θ(n^log7). Or that is T(n) = Θ(n^2.81).

Note:

1. The best available algorithm to multiply two nxn matrices is O(n^2.37)

2. To multiply two matrices of order mxn and nxp orders, we need m*n*p multiplications
and m*(n-1)*p additions.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Changing variables:-

Sometimes, a little algebraic manipulation can make an unknown recurrence similar to one we have
seen before.

n=2^m and m=logn

The new recurrence has the same solution: S(m)= O(mlogm) changing back from S(m)to T(n) we
obtain

Que (CLR):
Solution:

T(n)=Θ(logn)^log3

The recursion-tree method for solving recurrences

In a recursion tree, each node represents the cost of a single subproblem somewhere in the set of
recursive function invocations. We sum the costs within each level of the tree to obtain a set of
per-level costs, and then we sum all the per-level costs to determine the total cost of all levels of
the recursion.

For example, let us see how a recursion tree would provide a good guess for the recurrence
T (n) = 3T(n/4) + Θ(n^2)

Below figure shows how we derive the recursion tree for T(n) =3T(n/4) + c.n^2 For convenience, we
assume that n is an exact power of 4 so that all subproblem sizes are integers.

1. Part (a) of the figure shows T(n), which we expand in part (b) into an equivalent tree
representing the recurrence.
2. The cn^2 term at the root represents the cost at the top level of recursion, and the three
subtrees of the root represent the costs incurred by the subproblems of size n/4.
3. Part (c) shows this process carried one step further by expanding each node with cost T(n/4)
from part (b).
4. The cost for each of the three children of the root is c.(n/4)^2.
5. We continue expanding each node in the tree by breaking it into its constituent parts as
determined by the recurrence.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

In another example, Figure 4.6 shows the recursion tree for T(n)= T(n/3) + T(2n/3) + O(n)

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Intuitively, we expect the solution to the recurrence to be at most the number of levels times the
cost of each level, or O(c*n)*logn base(3/2) = O(nlogn)

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Que: - Use the master method to give tight asymptotic bounds for the following recurrences.

a. n^log2 base(4) = n^log4^(1/2) base(4) = theta(n^1/2)

b. theta(n^1/2 *logn)
c. theta(n) as n>n^log2 (base4)
d. theta(n^2)

Solution:-

a. n^loga base(b), hence theta(n^4)

b. n^log7 base(10) = n^0.85 < n hence theta(n)
c. theta(n^2logn)
d. n^log7 base(3) < n^2 hence, theta(n^2)
e. n^log7 (base2) > n^2 hence theta(n^log7 (base2))
f. n^log2 (base4) = theta(n^1/2*logn)
g. theta(n^3)
T(n) = T(n-4) + (n-2)^2 + n^2
T(n) = T(n-6) + (n-4)^2 + (n-2)^2 + n^2|
T(n) = T(n-8) + (n-6)^2 + (n-4)^2 + (n-2)^2 + n^2
.
.kth term
.
T(n) = T(n-2k) + (n-(2k-2))^2 + (n-(2k-4))^2 + …………….. (n-4)^2 +(n-2)^2 + n^2

Suppose 2k = n, k=n/2
=T(0) + 2^2 + 4^2 + 6^2 + 8^2 + ………………..(n-2)^2 + n^2
=1 + 2^2[ 1 + 2^2 + 3^2 + 4^2 + ………. k^2] (there are n/2 total terms)
=1 + 4[ k(k+1)(2k+1)/6] = k^3 or theta(n^3) as k=n/2

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

1/log6 = 1/log2*3 = 1/(log2 + log3) = log2 can be ignored

T(n) = 1/log2 + 1/log3 + 1/log4 + 1/log5 + ……………..1/log(n/2)

suppose we have been given a series

S=3+4+7+9+16+17.........100. (Assume there are 75 terms in this series)
In worst case if we can’t find any pattern, we can at least say that
3*75 <= S <= 100*75

Hence, n*(1/logn) <= T(n) <= n*(1/log2) So TC will be theta(n/logn) or O(n)

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Growth of Functions

O-notation (Big – oh notation)

When we have only an asymptotic upper bound, we use O-notation. For a given function g(n), we
denote by O(g(n)) the set of functions
O(g(n)) = { f(n): there contains positive c and n0 such that

We use O-notation to give an upper bound on a function, to within a constant factor.

Example:-

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

(small – oh condition)

(Small – omega)

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Comparing functions:-

1. Transitivity

2. Reflexivity

3. Symmetry

4. Transpose symmetry:

Because these properties hold for asymptotic notations, we can draw an analogy between the
asymptotic comparison of two functions f and g and the comparison of two real numbers a and b:

Note:-
1.

2. Not all functions are asymptotically comparable, That is, for two functions f(n) and g(n), it
may be the case that neither f(n)=O(g(n)) nor f(n)=omega(g(n)) holds.

Example: we can compare the function n and n^{1+sin n} since the value of sin n will oscillate b/w
0 and 1, and exponent of n^{1+sin n} will oscillate b/w 0 and 2.

Properties:-

1. Let f(n) and g(n) be asymptotically nonnegative functions, then

2. For any real constants a and b, where b > 0

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

3. Following property holds

Example: Is 2^(n+1) = O(2^n)? Is 2^2n = O(2^n)?

Monotonicity:-

Exponentials
1.
For all real a > 0, m, and n, we have the following identities:

2.

Logarithms:-
1. We shall use the following notations:

2. For all real a > 0, b > 0, c > 0, and n,

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Note: - Where, in each equation above, logarithm bases are not 1.

3. There is a simple series expansion for log(1 + x) when |x| < 1:

4. Following properties hold true

The iterated logarithm function

We define the iterated logarithm function as:

The iterated logarithm is a very slowly growing function:

Note:-
If f(n) and g(n) are monotonically increasing functions, then so are the functions f(n) + g(n) and
f(g(n)), and if f(n) and g(n) are in addition nonnegative, then f(n)*g(n) is monotonically increasing.

Example:-

Solution:
Suppose n = 2^128
log*(2^65536) = 5
log(log*n) = log(5) = 2 (approx)

log*(log2^65536) = log*(2^16)= 4

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Therefore, log*(logn) grows more quickly.

Relative asymptotic growths

Que Coreman 3.2

Solution:-

a.

b. n^k is a polynomial function, while c^n is an exponential function

hence n^k = o(c^n) or O(c^n)
or take log both sides
k*logn = n*logc k and c are constant factors
logn = o(n) (small-oh)

c. 2^n = 2^(n/2)
2^n > Root(2^n)

d. n^logc and c^logn

both equal functions as n^logc can be written as c^logn which equal to RHS
hence A = O(B), A=Omega(B) hence A=Theeta(B)

e. It’s a known property log(n!) = Theeta(nlogn)

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

f. 1 = o(1/n) (small-oh)

Que Coreman 3.3

Write the following functions in increasing order. And mention those functions which are
asymptotically equal to each other.

Solution-

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Important Properties:-

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

e^n > n*2^n

Ques 3-4 Coreman:

Let f(n) and g(n) be asymptotically positive functions. Prove or disprove each of the following
conjecture.

Solution:
a. False. Counterexample: n = O(n^2) but n^2 != O(n).
b. False. Counterexample: n + n^2 != O(n).

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

c. True

d.

e. False, counter example is f(n) = 1/n as 1/n != O(1/n^2)

f. True

g. False. Let f(n) = 2^2n then f(n/2) = 2^n and 2^2n != O(2^n)
h. True.

Que (GO)

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Note:- C is not growing because e^(n*constant).

Sort a nearly sorted Array (K-Sorted Array)

We have an unsorted array in which any element is misplaced no more than k position from its
correct position.

1. Insertion Sort the inner while loop won’t run more than k times hence T.C will be O(n*k)

2. Using Heap Data Structure

a. Take k+1 elements from unsorted array and create min-heap in O(k) time. We are
sure that this min heap of (k+1) elements will have smallest element.
b. Remove root element from min-heap O(logk) time and keep it in some temporary
array called as resulted array.
c. Next, insert another from unsorted array which is left with (n-k-1) elements into
min-heap in O(logk) time.
d. Now, min-heap will have second smallest element at the root, remove it, put it into
resulted array.
e. Continue until there is no element left in unsorted array

Time Complexity O(k) + O(n-k)*logk = O(nlogk)

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Part II Sorting and Order Statistics

Merge Sort

The merge sort algorithm closely follows the divide-and-conquer paradigm. Intuitively, it operates as
follows.
Divide: Divide the n-element sequence to be sorted into two subsequences of n/2 elements each.
Conquer: Sort the two subsequences recursively using merge sort.
Combine: Merge the two sorted subsequences to produce the sorted answer.

The recursion “bottoms out” when the sequence to be sorted has length 1, since every sequence of
length 1 is already in sorted order.
The key operation of the merge sort algorithm is the merging of two sorted sequences in the
“combine” step. We merge by calling an auxiliary procedure MERGE (A, p, q, r), where A is an array
and p, q, and r are indices into the array such that p <=q < r. The procedure assumes that the sub-
arrays A[p…q] and A[q+1….r] are in sorted order. It merges them to form a single sorted subarray
that replaces the current subarray A[p….r].

Note: - MERGE procedure takes time Θ(n), where n= r-p + 1 is the total number of elements being
merged.

Merge procedure consists steps of choosing the smaller of the two top elements of two sorted list.
Removing it from its list (which exposes a new top card), and placing this element into the output
list.
We repeat this step until one input sorted list is empty, and we insert remaining elements into
output list. Each basic step takes constant time.

The following pseudocode implements the merge procedure but with additional logic that avoids
having to check whether either list is empty in each basic step. We place on the bottom of each list a
sentinel element. We use the infinite as the sentential value. Since we know in advance that exactly
r - p + 1 cards will be placed onto the output pile, we can stop once we have performed that many
basic steps.

1. Line computes the length n1 of the sub array A[p………q]

2. Line 2 computes the length n2 of the subarray A[q+1 ……..r]
3. We create two array L and R (Left & Right) of lengths n1+1 and n2+1 respectively. Extra position
in each array will hold sentential element.
4. The for loops of line 4&5 copies the sub array A[p…..q] into L[1 …… n1]
5. The for loops of line 6&7 copies the sub array A[q+1…..r] into R[1 …… n2]
6. Line 8 and 9 put the sentinels at the ends of the arrays L and R.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

From lines 12 to 17, a loop runs from p to r, it selects minimum of two elements on the top of list
and insert into original list A

We can now use the MERGE procedure as a subroutine in the merge sort algorithm.
1. The procedure MERGE-SORT(A, p, r) sorts the elements in the subarray A[p…..r].
2. If p>=r, the sub array has at most one element and is therefore already sorted.
3. Otherwise, the divide step simply computes an index q that partitions A[p….r] into two
subarrays: A[p….q], containing ceil[n/2] elements and A[q+1 ….r] containing floor[n/2]
elements.

Note: - To sort the entire sequence A = <A[1], A[2], ……., A[n]>, we make the initial call
MERGE-SORT(A, 1, A.length), where once again A.length=n.

The algorithm consists of merging pairs of 1-item sequences to form sorted sequences of length 2,
merging pairs of sequences of length 2 to form sorted sequences of length 4, and so on, until two
sequences of length n/2 are merged to form the final sorted sequence of length n.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Merge Procedure

Divide and Conquer recurrence relation Analysis:

Let T(n) be the running time on a problem of size n.

1. If the problem size is small enough, say n <= c for some constant c, the straightforward solution
takes constant time, which we write as Θ(1).
2. Suppose that division of the problem yields a subproblems, each of which is 1/b the size of the
original.
3. It takes time T(n/b) to solve one subproblem of size n/b, and so it takes time aT(n/b) to solve a
of them.
4. If we take D(n) time to divide the problem into subproblems and C(n) time to combine the
solutions to the subproblems into the solution to the original problem. We get the recurrence

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Analysis of merge sort

The worst-case running time of merge sort on n numbers. Merge sort on just one element takes
constant time. When we have n > 1 elements, we break down the running time as follows.

Divide: The divide step just computes the middle of the subarray, which takes constant time. Thus,
D(n) = Θ(1).
Conquer: We recursively solve two subproblems, each of size n/2, which contributes 2T(n/2) to the
running time.
Combine: We have already noted that the MERGE procedure on an n-element subarray takes time
Θ(n), and so C(n) = Θ(n)

When we add the functions D(n) and C(n) for the merge sort analysis, we are adding a function that
is Θ(n) and a function that is Θ(1). This sum is a linear function of n, that is Θ(n).

T.C is Θ(nlogn)

Merge Sort Properties:-

1. Merge sort is stable and not in-place, comparison-based algorithm.

2. It’s use for sorting linked-list in O(nlogn).
3. It’s useful in external Sorting such as database table don’t fit in memory, need to sort on
disk.
4. Useful in Inversion Count Problem.
5. sort requires Ω(n) auxiliary space.

Binary Search:-

The following recursive algorithm gives the desired result when called with a = 1 and b = n.

m is the middle element, v is the search key. If v is equal to middle element, return it. Else if v < m
then apply BS from a to m else v>m then apply BS from m+1 to b.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

We can see that the while loop gets run at most O(n) times, as the quantity j – i starts at n -1 and
decreases at each step.

Insertion sort on small arrays in merge sort

Although merge sort runs in Θ(nlogn) worst-case time and insertion sort runs in Θ(n^2) worst-case
time, the constant factors in insertion sort can make it faster in practice for small problem sizes on
many machines.
In merge sort, we can user insertion sort when subproblems become sufficiently small. Consider a
modification to merge sort in which n/k subsists of length k are sorted using insertion sort and then
merged using the standard merging mechanism, where k is a value to be determined.
Questions:

Solutions:

b. Start merging it at the level in which each array has size at most k. This means that the depth of
the merge tree is log(n) - log(k) = log(n/k). Each level of merging is still time cn, so putting it
Together, the merging takes time Θ (nlog(n/k)).

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

c. If order for Θ(nk + nlog(n/k)) = Θ(nlogn), either nk=nlogn or nlog(n/k) = nlogn. From these two
possibilities we know the largest asymptotic value for k is Θ(logn)

d. A constant choice of k is optimal.

Heap Sort
1. Like merge sort, but unlike insertion sort, heapsort’s running time is O(nlogn)
2. Like insertion sort, but unlike merge sort, heapsort sorts in place. Only a constant number of
array elements are stored outside the input array at any time.
3. Thus, heapsort combines the better attributes of the two sorting algorithms.

Heaps:
The (binary) heap data structure is an array object that we can view as a nearly complete binary
tree.
Each node of the tree corresponds to an element of the array. The tree is completely filled on all
levels except possibly the lowest, which is filled from the left up to a point. An array A that
represents a heap is an object with two attributes: A.length, which gives the number of elements in
array and A.heap-size, which represents how many elements in the heap are stored within array A.

That is, although A[1 . . A.length] may contain elements, only the elements in A[1 . . A.heap-size],
where 0 <= A.heap-size <= A.length, are valid elements of the heap. The root of the tree is A[1] and
given the index i of a node, we can easily compute the indices of its parent, left child, and right
child.

A max-heap viewed as (a) a binary tree and (b) an array

There are two kinds of binary heaps: max-heaps and min-heaps. In both kinds, the values in the
nodes satisfy a heap property.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Max-Heap: In a max-heap, the max-heap property is that for every node i other than the root,

That is, the value of a node is at most the value of its parent. The largest element in a max-heap is
stored at the root, and the subtree rooted at a node contains values no larger than that contained at
the node itself.

Min-Heap: The min-heap property is that for every node i other than the root:
The smallest element is at the root.
Note: -
a. Heapsort algorithm uses max-heaps
b. Min-heaps commonly implement priority queue.

Height of a Node: number of edges on the longest simple downward path from the node to a leaf.
Height of Heap: height of root node.

Note: -

1. Since a heap of n elements is based on a complete binary tree, its height is Θ(logn).
2. The basic operations on heaps run in time at most proportional to the height of the tree and
thus take Θ(logn) time.

• The MAX-HEAPIFY procedure, which runs in O(logn) time, is the key to maintaining the max-
heap property.
• The BUILD-MAX-HEAP procedure, which runs in linear time O(n), produces a max-heap from
an unordered input array.
• The HEAPSORT procedure, which runs in O(nlogn) time, sorts an array in place.
• The MAX-HEAP-INSERT, HEAP-EXTRACT-MAX, HEAP-INCREASE-KEY, and HEAP-MAXIMUM
procedures, which run in O(log n) time, allow the heap data structure to implement a
priority queue.

Note: - in a heap tree the leaves are the nodes indexed by

Que 6.1-1 What are the minimum and maximum numbers of elements in a heap of height h?
Solution:
Maximum elements when tree is completely filled at all levels i.e. 2^(h+1) -1
Minimum elements when there is only one node at last level i.e. 2^h

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Que 6.1-2
Solution: n = (2^h) – 1 + k, Then the heap consists of a complete binary tree of height h -1, along
with k additional leaves along the bottom. The height of the root is the length of the longest simple
path to one of these k leaves, which must have length h.
It is clear from the way we defined h that h =

Que 6.1-4 where in a max-heap might the smallest element reside, assuming that all elements are
distinct?
Solution:
The smallest element must be a leaf node. Suppose that node x contains the smallest element and x
is not a leaf. Let y denote a child node of x. By the max-heap property, the value of x is greater than
or equal to the value of y. Since the elements of the heap are distinct, the inequality is strict.

Que 6.1-5 Is an array that is in sorted order a min-heap?

Yes, it is. The index of a child is always greater than the index of the parent, so the heap property is
satisfied at each vertex.

Maintaining the heap property

In order to maintain the max-heap property, we call the procedure MAX-HEAPIFY. Its inputs are an
array A and an index i into the array. When it is called, MAXHEAPIFY assumes that
1. The binary trees rooted at LEFT[i] and RIGHT[i] are maxheaps, but that A[i] might be smaller
than its children, thus violating the max-heap property.
2. MAX-HEAPIFY lets the value at A[i] “float down” in the max-heap so that the subtree rooted at
index i obeys the max-heap property

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Line 3 & 4: index l and r must be less than equal to total size of heap.

1. At each step, the largest of the elements A[i], A[left], and A[right] is determined, and its
index is stored in largest.
2. If A[i] is largest, then the subtree rooted at node i is already a max-heap and the procedure
terminates.
3. Otherwise, one of the two children has the largest element, and A[i] is swapped with
A[largest], which causes node i and its children to satisfy the max-heap property.
4. Subtree rooted at largest might violate the max-heap property. Consequently, we call MAX-
HEAPIFY recursively on that subtree.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

The running time of MAX-HEAPIFY on a subtree of size n rooted at a given node i is the Θ(1) time to
fix up the relationships among the elements A[i], A[LEFT], and A[RIGHT], plus the time to run MAX-
HEAPIFY on a subtree rooted at one of the children of node i.
The children’s subtrees each have size at most 2n/3—the worst case occurs when the bottom level
of the tree is exactly half full and therefore we can describe the running time of MAX-HEAPIFY by the
recurrence.

a=1, b=3/2, n^(loga (baseb)) = log 1 base(3/2) hence n^0 = 1, thus T.C Θ(logn)

Alternatively, we can characterize the running time of MAXHEAPIFY on a node of height h as O(h).

LST contains 2n/3 nodes

BUILD-MAX-HEAP

6.3-2 why do we want the loop index i in line 2 of BUILD-MAX-HEAP to decrease from

Ans: If we had started at 1, we wouldn't be able to guarantee that the max-heap property is
maintained.
For example, if the array A is given by [2, 1, 1, 3] then MAX-HEAPIFY won't exchange 2 with either of
its children, both 1's. But, when MAX-HEAPIFY is called on the left child, 1, it will swap 1 with 3.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

If we start at root, Max-heapify fails

Build Max-Heap Procedure

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

The heapsort algorithm

The heapsort algorithm starts by using BUILD-MAX-HEAP to build a max-heap on the input array
A[1…….n] where n=A.length.

1. Since the maximum element of the array is stored at the root A[1], we can put it into its correct
final position by exchanging it with A[n].
2. If we discard nth node from the heap, and we can do so by simply decrementing A.heap-size – 1;
3. We observe that the children of the root remain max-heaps, but the new root element might
violate the max-heap property.
4. To restore the max-heap property, MAX-HEAPIFY(A, 1) is called, which leaves a max-heap in
A[1……. N-1].
5. The heapsort algorithm then repeats this process for the max-heap of size n - 1 down to a heap
of size 2. Following is example of heapsort algorithm.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

The HEAPSORT procedure takes time O(nlogn), since the call to BUILD-MAXHEAP takes time O(n)
and each of the n - 1 calls to MAX-HEAPIFY takes time O(logn).

Properties:-
1. Best, average and worst case T.C is O(nlogn)
2. Merge Sort outperforms heap sort in most of the practical cases.
3. Heapsort is in-place sorting algorithm
4. Heap Sort is not stable.
5. Heapsort can be adapted to operate on doubly linked lists with only O(1) extra space.

Que 6.4-3 What is the running time of HEAPSORT on an array A of length n that is already sorted in
increasing order? What about decreasing order?
Solution:
The running time of HEAPSORT on an array of length n that is already sorted in increasing order is
Θ(nlogn) because even though it is already sorted, it will be transformed back into a max-heap in
O(n) time and then sorted.

Same goes for decreasing order. BUILD-MAX-HEAP will be faster (by a constant factor), but the
computation time will be dominated by the loop in HEAPSORT, which is Θ(nlogn).

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Priority queues

As with heaps, priority queues come in two forms: max-priority queues and min-priority queues.
We will focus here on how to implement max-priority queues, which are based on maxheaps.

A priority queue is a data structure for maintaining a set S of elements, each with an associated
value called a key. A max-priority queue supports the following operations:

1. INSERT(S, x) inserts the element x into the set S, which is equivalent to the operation S = S U {x}.
2. MAXIMUM(S) returns the element of S with the largest key.
3. EXTRACT-MAX(S) removes and returns the element of S with the largest key.
4. INCREASE-KEY(S, x, k) increases the value of element x’s key to the new value k, which is
assumed to be at least as large as x’s current key value.

Note: -
1. We can use max-priority queues to schedule jobs on a shared computer.
2. Min-priority queue supports the operations INSERT, MINIMUM, EXTRACT-MIN, & DECREASEKEY.
3. A min-priority queue can be used in an event-driven simulator. The items in the queue are
events to be simulated, each with an associated time of occurrence that serves as its key. The
events must be simulated in order of their time of occurrence.

Now we discuss how to implement the operations of a max-priority queue.

1. Θ(1) time, it returns root element.

2. Θ(logn) time. It extracts root element, replace with last element and apply max-heapify.

3. It updates the key A[i] to its new value. Because increasing the key of A[i] might violate the max-
heap property. HEAP-INCREASE-KEY on an n-element heap is O(log n). It repeatedly compares an
element to its parent, exchanging their keys and continuing if the element’s key is larger.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

4. It first expands the max-heap by adding to the tree a new leaf whose key is - infinity. Then it
calls HEAP-INCREASE-KEY to set the key of this new node to its correct value and maintain the
max-heap property. The running time of MAX-HEAP-INSERT on an n-element heap is O(logn).

Que 6.5-8
The operation HEAP-DELETE(A, i) deletes the item in node i from heap A. Give an implementation of
HEAP-DELETE that runs in O(logn) time for an n-element max-heap.
Solution
Replace the node to be deleted by the last node of the heap. Update the size of the heap, then call
MAX-HEAPIFY to move that node into its proper position. O(logn)

Que 6.5-9 Give an O(nlogk) time algorithm to merge k sorted lists into one sorted list, where n is the
total number of elements in all the input lists.
Solution:
There are n elements, and k sorted list of n/k elements each.
1. Construct a min heap from the heads of each of the k lists.
2. To find the next element in the sorted array, extract the minimum element (in Olog(k) time)
3. Then, add to the heap the next element from the shorter list from which the extracted element
originally came i.e O(logk)
4. Time to find the whole list, you need O(nlog(k)) total steps.

Young tableaus:
An m x n Young tableau is an m x n matrix such that the entries of each row are in sorted order from
left to right and the entries of each column are in sorted order from top to bottom. Some of the
entries of a Young tableau may be infinity, which we treat as non-existent elements.

Properties:-
1. To insert a new element into a non-full m x n Young tableau in O(m+n) time.
2. To determine whether a given number is stored in a given m x n Young tableau is O(m+n).
3. Using no other sorting method as a subroutine, an n x n Young tableau uses to sort n^2 numbers
in O(n^3) time.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Quick Sort
Description of quicksort

Quicksort, like merge sort, applies the divide-and-conquer paradigm.

Divide: Partition (rearrange) the array A[p……r] into two (possibly empty) subarrays A[p……q-1] and
A[q+1…….r] such that each element of A[p……q-1] is less than or equal to A[q], which is, in turn,
less than or equal to each element of A[q+1…….r]. Compute the index q as part of this partitioning
procedure.

Conquer: Sort the two subarrays A[p……q-1] and A[q+1…….r] by recursive calls to quicksort.

Combine: Because the subarrays are already sorted, no work is needed to combine them: the entire
array A[p……r] is now sorted.

The following procedure implements quicksort( ArrayName, StartIndex, length)

Note- To sort an entire array A, the initial call is QUICKSORT(A, 1, A.length).

Partitioning the array

The key to the algorithm is the PARTITION procedure, which rearranges the subarray A[p…r] in place.

1. We’re choosing the last element of the array A[p…r] as pivot element.
2. i is assigned to 0 (p – 1, array starts with 1)
3. j for loop starts from p (first element) and goes till r – 1 (n-1th element)
4. If A[j] is smaller or equal to pivot element then increase i.
5. Exchange A[i] and A[j]
6. After the end of for loop, exchange A[i+1] and pivot element A[r]
7. Return i+1 as the position of pivot element.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Partition Algorithm

Note: -The running time of PARTITION on the subarray A[p….r] is Θ(n) , where n= r – p + 1.

Solution:

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

If all elements in the array have the same value, PARTITION returns r.
To (q should be middle of array) when all elements
are same value modify line 4 of the partition algorithm to say this:
If A[j] <= x and j(mod2) = (p + 1)(mod2)
This causes the algorithm to treat half of the instances of the same value to count as less than, and
the other half to count as greater than.

Performance of quicksort
The running time of quicksort depends on whether the partitioning is balanced or unbalanced,
which in turn depends on which elements are used for partitioning.
1. If the partitioning is balanced, the algorithm runs asymptotically as fast as merge sort.
2. If the partitioning is unbalanced, the algorithm runs asymptotically as slow as Insertion Sort.

Worst-case partitioning [T(n) = Θ(n^2)]

The worst-case behavior for quicksort occurs when the partitioning routine produces one
subproblem with n - 1 elements and one with 0 elements. Let us assume that this unbalanced
partitioning arises in each recursive call. The partitioning costs Θ(n) time. The recurrence for the
running time is

Note: - If array is already sorted then Q.S takes Θ (n^2) and Insertion Sort takes O(n)

Best-case partitioning [Θ(nlogn)]

Balanced partitioning the average-case running time of quicksort is much closer to the best case
than to the worst case.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

a. T(n) = T(9n/10) + T(n/10) + cn

b. T(n) = T(99n/100) + T(n/100) + cn

The running time is therefore O(nlogn) whenever the split has constant proportionality.

Intuition for the average case

Good split and bad split

In the average case, PARTITION produces a mix of “good” and “bad” splits. In a recursion tree for an
average-case execution of PARTITION, the good and bad splits are distributed randomly throughout
the tree.
At the root of the tree, cost is n for partitioning, and the subarrays produced have sizes n - 1 and 0
the worst case. At the next level, the subarray of size n - 1 undergoes best-case partitioning into
subarrays of size (n-1)/2 -1 and (n – 1)/2.

Que 7.2-2 What is the running time of QUICKSORT when all elements of array A have the same
value?
Solution:
The running time of QUICKSORT on an array in which every element has the same value is n^2. This
is because the partition will always occur at the last position of the array.

A randomized version of quicksort

Instead of always using A[r] as the pivot, we will select a randomly chosen element from the
subarray A[p…r] We do so by first exchanging element A[r] with an element chosen at random
from A[p…r] By randomly sampling the range p….r, we ensure that the pivot element x = A[r] is
equally likely to be any of the r - p + 1 elements in the subarray. Because we randomly choose the
pivot element, we expect the split of the input array to be reasonably well balanced on average.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

The new quicksort calls RANDOMIZED-PARTITION in place of PARTITION:

Analysis of Randomized Quick Sort:-

1. Expected T.C for Randomized Q.S is Θ(nlogn)

2. Worst case T.C is Still O(n^2) when either all elements are same or the randomized pivot
selector happens to select smallest element in a row n time. Though this possibility is only 1/n!

Note: T(n) = T(q) + T(n – q – 1) + Θ(n), T(n) = omega(n^2)

Comparison based Sorting Algoirthms

All the sorting algorithms introduced thus far are comparison sorts. The sorted order they determine
is based only on comparisons between the input elements. We call such sorting algorithms
comparison sorts.

Any comparison sort must make omega(nlogn) comparisons in the worst case to sort n elements.

Note: - Heapsort and merge sort are asymptotically optimal comparison sorts.

The decision-tree model:

A decision tree is a full binary tree that represents the comparisons between elements that are
performed by a particular sorting algorithm operating on an input of a given size.

With n elements, there are total n! Permutations possible. Each leaf node in decision represents 1 of
n! Permutations.

Number of comparisons to sort n elements: Ceil (logn!)

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Sorting in Linear Time

Counting Sort (Stable Sort)

Counting sort assumes that each of the n input elements is an integer in the range 0 to k, for some
integer k. When k = O(n), the sort runs in Θ(n) time.

Counting sort determines, for each input element x, the number of elements less than x. It uses this
information to place element x directly into its position in the output array.

In the code for counting sort, we assume that

1. input is an array A[1…n], and thus A.length = n.
2. We require two other arrays: the array B[1…n] holds the sorted output.
3. Array C[0..k] provides temporary working storage, size of this array is k+1.

K = Max element in array

1. C[i]: How many times the element i appears in array A. For example 0 appeared two times in
array A, hence C[0]=2, or 3 is three times in array A.

2. Start from second element in array C and add the value of that cell C[i] with the value of
previous cell C[i-1].

Start from the last element in array A.

1. Last element in array A is 3, in array C, it is C[3]=7.
2. We will store 3 at position 7 in array B, and we will decrease value of C[3] by 1.

(3 is stored at 7th position)

(C[3] is decreased by 1)

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

3. Repeat the procedure with second last element.

a. Second last element is 0 in array A, C[0] = 2, hence 0 will be stored at B[2].

b. C[0] is decremented by 1.

4. Repeat the procedure with 3rd last element

a. 3rd last element in A is 3, C[3] = 6, hence 3 will be stored at B[6]

b. C[3] will be decreased by 1

5. At the end, Array will have elements in sorted manner:

Overall time is Θ(k+n). In practice, we usually use counting sort when we have k = O(n).

Properties:-
1. Counting sort is that it is stable.
2. Counting sort is not in-place.
3. Counting sort is often used as a subroutine in radix sort.

Radix sort
Radix sort solves the problem by sorting on the least significant digit first. If LSB is same then write
the no. first which appears first in the input array.
The process continues until the cards have been sorted on all d digits. To sort d digit numbers, d
passes are required.

The above procedure assumes that each element in the n-element array A has d digits, where digit
1 is the lowest-order digit and digit d is the highest-order digit.

Given n d-digit numbers in which each digit can take on up to k possible values, RADIX-SORT
correctly sorts these numbers in Θ(d(n+k)) time if the stable sort it uses takes Θ(n+k) time.

Note: - When d is constant and k = O(n) we can make radix sort run in linear time.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Note:- We can sort an array of integers with range from 1 to n^c, if the numbers are represented
in base n, each number will have c digits.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Elementary Data Structures

Stacks

We can implement a stack of at most n elements with an array S[1…n]. The array has an attribute
S.top that indexes the most recently inserted element.

The stack consists of elements S[1… S.top], where S[1] is the element at the bottom of the stack
and S[Top] is the element at the top.

When S.top = 0, the stack contains no elements and is empty.

We can implement each of the stack operations with just a few lines of code:

PUSH First increment TOP and then insert the TOP.

POP First copy the element and then decrement the TOP

Each of the three stack operations takes O(1) time.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Queues

DEQUEUE takes no element argument. The FIFO property of a queue causes it to operate like a line
of customers waiting to pay a cashier. The queue has a head (front) and a tail (rear).
1. When an element is enqueued, it takes its place at the tail of the queue, just as a newly arriving
customer takes a place at the end of the line.
2. The element dequeued is always the one at the head of the queue, like the customer at the
head of the line who has waited the longest.

Figure 10.2 shows one way to implement a queue of at most n - 1 elements using an array Q[1..n]

1. The queue has an attribute Q.head points to, its head.

2. The attribute Q.tail indexes the next location at which a newly arriving element will be
inserted into the queue
3. The elements in the queue reside in locations Q.head; Q.head+1; ….. Q.tail-1.
4. Location 1 immediately follows location n in a circular order.
5. ENQUEUE and DEQUEUE operations, Each operation takes O(1) time.

Note: -
1. Q.head = Q.tail, the queue is empty.
2. Initially, we have Q.head = Q.tail = 1.
3. When Q.head = Q.tail +1, the queue is full.

In our procedures ENQUEUE and DEQUEUE, we have omitted the error checking for underflow and
overflow. The pseudocode assumes that n = Q.length.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Queues Underflow and Overflow Conditions:

Enqueue: Insert the element and then increment the pointer.

Dequeue: Copy the element and then increment the pointer.

If (Q.head == Q.tail + 1 or Q.head == 1 && Q.tail == Q.length)

then “error overflow" ( 1 cell is wasted )

if Q.tail == Q.head then

error “underflow"

Que 10.1-2 Explain how to implement two stacks in one array A[1…n] in such a way that neither
stack overflows unless the total number of elements in both stacks together is n. The PUSH and POP
operations should run in O(1) time.
Solution

Array is A[1…n] starting from 1 to n.

We will call the stacks T and R. Initially, set T.top = 0 and R.top = n + 1.
1. Stack T uses the first part of the array and stack.
2. R uses the last part of the array.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

3. In stack T, the top is the rightmost element of T.

4. In stack R, the top is the leftmost element of R.

Initially position of T.Top and R.Top

Note- S is a stack pointer formal parameter to function PUSH.Stack name will be provided as
actual parameter while calling the function PUSH

1. Overflow condition

If S = T then increment TOP and insert the element.

2. Overflow condition if T.Top = R.Top-1

If S=R then decrement the TOP and insert the element.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Que 10.1-6 Show how to implement a queue using two stacks. Analyse the running time of the
queue operations.
Solution:

1. The operation enqueue will be same as pushing an element on to stack 1. This operation is O(1).
2. To dequeue, we pop an element from stack 2. If stack 2 is empty, for each element in stack 1 we
pop it off, then push it on to stack 2. Finally, pop the top item from stack 2. This operation is
O(n) in the worst case.

Que 10.1.7 Show how to implement a stack using two queues. Analyze the running time of the stack
operations.
Solution:
The following is a way of implementing a stack using two queues, where
1. Pop takes linear time O(n)
2. Push takes constant time O(1)

Push and Pop are not fixed to any particular queue.

PUSH( )
1. Enqueue items in queue 1.

POP()
1. If queue contains more than 1 element, dequeue elements from Queue1 and place them on
Queue2, except the last element. Return the single element left in Queue1.
2. Then switch the name of queues queue1 and queue2.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Linked lists:

A linked list is a data structure in which the objects are arranged in a linear order. Unlike an array,
however, in which the linear order is determined by the array indices, the order in a linked list is
determined by a pointer in each object.

Doubly Linked List

L is an object with an attribute key and two other pointer attributes: next and pre. The object may
also contain other satellite data.

1. Given an element x in the list, x.next points to its successor in the linked list, and x.pre points to
its predecessor.
2. If x.pre = NIL, the element x has no predecessor and is therefore the first element, or head, of
the list.
3. If x.next = NIL, the element x has no successor and is therefore the last element, or tail, of the
list.
4. An attribute L.head points to the first element of the list. If L.head = NIL, the list is empty.

1. If a list is single linked list then we omit pre pointer in each element.
2. If a list is sorted, minimum element is the head and maximum element is the tail.
3. If a list is unsorted, the elements can appear in any order.
4. In a circular list, the pre pointer of the head of the list points to the tail, and the next pointer
of the tail of the list points to the head.

Searching a linked list

The procedure LIST-SEARCH(L, k) finds the first element with key k in list L by a simple linear search,
returning a pointer to this element. If no object with key k appears in the list, then the procedure
returns NIL. The LIST-SEARCH procedure takes Θ(n) time in the worst case.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Inserting into a linked list O(1)

Given an element x whose key attribute has already been set, the LIST-INSERT procedure insert x
onto the front of the linked list in O(1) time.

X is the node to be inserted.

1. X.next will store the pointer, pointed by pointer head.
2. If the linked list is not nil, pre pointer of the node previously pointed by header, will store
the address of x. New node will be pointing to the node previously pointed by header.
3. If Linked list was nil, then Head will point to node x, and pre pointer of x will contain null.

Deleting from a linked list

The procedure LIST-DELETE removes an element x from a linked list L. It must be given a pointer to x,
and it then “splices” x out of the list by updating pointers. If we wish to delete an element with a
given key, we must first call LIST-SEARCH to retrieve a pointer to the element.

X is pointing to the node that needs to be deleted.

1. If x is not the first node, then last node to the node x will point to the next node.
2. If x is the first node in the linked list then head will be pointing to the next node.
3. If x is not the last node then pre pointer of the next node to x node will be pointing to the
previous node to the x node.

Note:-
1. O(1) if pointer is given
2. Θ(n) if we wish to delete a node with given key.

Que 10.2-2 Implement a stack using a singly linked list L. The operations PUSH and POP should still
take O(1) time.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Solution:
The element which is pushed at the end will be popped first. We insert element at front of the linked
list hence we will pop element from the front of the linked list only.
1. The PUSH(L, x) operation is exactly the same as LIST-INSERT(L, x).
2. The POP operation sets x equal to L.head, calls LIST-DELETE(L, L.head), then returns x.

Note - Insertion and deletion both are at the front of the linked list.

Que 10.2-3 Implement a queue by a singly linked list L. The operations ENQUEUE and DEQUEUE
should still take O(1) time.
Solution:-
FIFO, hence we will perform dequeue at the start of Linked list, and will perform enqueue operation
at the end of linked list.
Hence, In addition to the head, also keep a pointer to the last node of linked list.
1. To enqueue, insert the element after the last element of the list, and set it to be the new
last element.
2. To dequeue, delete the first element of the list and return it.

Que 10.2-7 Give a Θ(n) time non-recursive procedure that reverses a singly linked list of n elements.
The procedure should use no more than constant storage beyond that needed for the list itself.
Solution

We are incrementing a and b to next nodes one by one.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Comparisons among lists

Note: - x is the given pointer, pointing to the required node.

Explanation:
1. To search a given key, we have to traverse whole linked list, hence, O(n)
2. To insert into unsorted linked list, we make insertion at the head of linked list O(1)
3. To insert into sorted list, we have to find correct place
4. Deletion in single linked list, it’s O(n) because if we are asked to delete last node we have to
traverse the whole linked list, though we are given a pointer pointing to the last node.
5. Deletion can happen in O(1) in doubly linked list because last node has pointer to second last
node.
6. Successor (L, x) in sorted list, next node will be successor, and we are given a pointer
pointing to the node whose successor to be found.
7. Predecessor(L, x) is linear in single linked list because there is no pre pointer. In sorted
doubly linked list the last node key will be predecessor.
8. Maximum will O(n), as we are not maintaining any tail pointer.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Hash Tables

Many applications require a dynamic set that supports only the dictionary operations INSERT,
SEARCH, and DELETE.
A hash table is an effective data structure for implementing dictionaries.

Although searching for an element in a hash table can take as long as searching for an element in a
linked list Θ(n) time in the worst case.

Note: - Under reasonable assumptions, average time to search for an element in a hash table is O(1).

Properties:
1. Hashing is an extremely effective and practical technique: the basic dictionary operations
require only O(1) time on the average.
2. “Perfect hashing” can support searches in O(1) worst case time, when the set of keys being
stored is static (that is, when the set of keys never changes once stored).

Direct-address tables

Direct addressing is a simple technique that works well when the universe U of keys is reasonably
small.
Suppose that an application needs a dynamic set in which each element has a key drawn from the
universe U = {0,1, ….., m-1} where m is not too large. We shall assume that no two elements have
the same key.

To represent the dynamic set, we use an array, or direct-address table, denoted by T[0, …., m-1], in
which each position, or slot, corresponds to a key in the universe U.

Each key in the universe U = {0, 1, … , 9} corresponds to an index in the table. The set K = {2, 3, 5, 8}
of actual keys determines the slots in the table that contain pointers to elements.

For some applications, the direct-address table itself can hold the elements in the dynamic set.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Hash tables
With direct addressing, an element with key k is stored in slot k. With hashing, this element is stored
in slot h(k); that is, we use a hash function h to compute the slot from the key k.

Two keys may hash to the same slot. We call this situation a collision.

Collision resolution by chaining

In chaining, we place all the elements that hash to the same slot into the same linked list.

Collision Resolution by chaining

1. The worst-case running time for insertion is O(1). The insertion procedure is fast in part because
it assumes that the element x being inserted is not already present in the table.
2. For searching, the worst case running time is proportional to the length of the list.
3. We can delete an element in O(1) time if the lists are doubly linked, Note that CHAINED-HASH-
DELETE takes as input an element x and not its key k, so that we don’t have to search for x first.
4. With singly linked lists, both deletion and searching would have the same asymptotic running
times.

Analysis of hashing with chaining

Given a hash table T with m slots that stores n elements:
Load factor
That is, the average number of elements stored in a chain. Which can be less than, equal to, or
greater than 1.

Note: - The worst-case behavior of hashing with chaining is terrible: all n keys hash to the same slot,
creating a list of length n. The worst-case time for searching is thus Θ(n) plus the time to compute
the hash function.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Universal hashing

Any fixed hash function is vulnerable to such terrible worst-case behavior; the only effective way to
improve the situation is to choose the hash function randomly in a way that is independent of the
keys that are actually going to be stored. This approach, called universal hashing.

Open addressing

In open addressing, all elements occupy the hash table itself. That is, each table entry contains
either an element of the dynamic set or NIL. When searching for an element, we systematically
examine table slots until either we find the desired element or we have ascertained that the
element is not in the table. No elements are stored outside the table, unlike in chaining.

Linear probing: -
h(k, i) = (h(k) + i) mod m

Linear probing is easy to implement, but it suffers from a problem known as primary clustering.
Long runs of occupied slots build up, increasing the average search time.

Quadratic probing

Quadratic probing uses a hash function of the form:

h(k, i) = ( h(k) + c1*i + c2*i^2) mod m
Where c1 and c2 are positive auxiliary constants, and i = 0, 1, 2, ……., m-1.

This method works much better than linear probing. This property leads to a milder form of
clustering, called secondary clustering. If two keys have the same initial probe position, then their
probe sequences are the same.

Double hashing
Double hashing offers one of the best methods available for open addressing. Double hashing uses a
hash function of the form:
h(k, i) = (h1(k) + i*h2(k)) mod m

Where both h1 and h2 are auxiliary hash functions.

Analysis of open-address hashing

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Perfect hashing
Hashing technique perfect hashing if O(1) memory accesses are required to perform a search in the
worst case.

Binary Search Trees

The search tree data structure supports many dynamic-set operations, including SEARCH,
MINIMUM, MAXIMUM, PREDECESSOR, SUCCESSOR, INSERT, and DELETE.

1. Basic operations on a binary search tree take time proportional to the height of the tree.
2. For a complete binary tree with n nodes, such operations run in Θ(n) worst-case time.
3. If the tree is a linear chain of n nodes, the same operations take Θ(n) worst-case time.
4. Expected height of a randomly built binary search tree is O(logn). So that basic dynamic-set
Operations on such a tree take Θ(logn) time on average.

The keys in a binary search tree are always stored in such a way as to satisfy the binary-search-tree
property.
Let x be a node in a binary search tree. If y is a node in the left subtree of x, then y.key <= x.key. If y
is a node in the right subtree of x, then y.key >= x.key.

1. An inorder tree walk print out all the keys in a binary search tree in sorted order.
2. A preorder tree walk prints the root before the values in either subtree.
3. A postorder tree walk prints the root after the values in its subtrees.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Inorder Tree Walk

To use the following procedure to print all the elements in a binary search tree T, we call
INORDER-TREE-WALK(T.root). It takes Θ(n).

It takes Θ(n) time to walk an n-node binary search tree, since after the initial call, the procedure calls
itself recursively exactly twice for each node in the tree—once for its left child and once for its right
child.

For n>0, suppose that INORDER-TREE-WALK is called on a node x whose left subtree has k nodes and
right subtree has n-k-1 nodes, time to perform INORDER-TREE-WALK(x) is

T(n) = T(k) + T(n-k+1) + c

Querying a binary search tree

1. Searching O(h)
Given a pointer to the root of the tree and a key k, TREE-SEARCH returns a pointer to a node with
key k if one exists; otherwise, it returns NIL.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

The running time of TREE-SEARCH is O(h), where h is the height of the tree.

The iterative version is more efficient.

Minimum and maximum O(h)

We can always find an element in a binary search tree whose key is a minimum by following left
child pointers from the root until we encounter a NIL.

The following procedure returns a pointer to the minimum element in the subtree rooted at a given
node x.

If a node x has no left subtree, then since every key in the right subtree of x is at least as
Large as x.key, the minimum key in the subtree rooted at x is x.key. Following procedure is to find
the maximum element in a B.S.T

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Successor and predecessor O(h)

Successor of a node x is the node with the smallest key greater than x.key.

1. A node has right subtree, then find minimum element from RST.
2. If a node has not a right sub tree, then climb up. Successor of such node will be its lowest
ancestor whose left child is also an ancestor.

1. If a node has left subtree, find the maximum element in LST will be inorder predecessor.
2. If left sub tree doesn’t exist then in-order predecessor is one of the ancestor of it. We can move
up towards root until we encounter a node which is right child of its parent.

Insertion and deletion O(h)

The procedure takes a node z for which z.key=v, z.left=NIL, and z.right=NIL It modifies T and some of
the attributes of z in such a way that it inserts z into an appropriate position in the tree.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Inserting an item with key 13 into a binary search tre

Deletion
The overall strategy for deleting a node z from a binary search tree T has three basic cases:
1. If z has no children, then we simply remove it by modifying its parent to replace z with NIL as its
child.
2. If z has just one child, then we elevate that child to take z’s position in the tree by modifying z’s
parent to replace z by z’s child.
3. If z has two children, then we find z’s successor y, which must be in z’s right subtree and have y
take z’s position in the tree. The rest of z’s original right subtree becomes y’s new right subtree,
and z’s left subtree becomes y’s new left subtree.

Theorem 12.4 the expected height of a randomly built binary search tree on n distinct keys is
O(logn).

Number of Binary trees possible with n nodes

no. of BST = no. of unlabelled B.T = 2nCn/(n+1)

no. of labelled trees = (no. of unlabelled trees)*n!

T(n) = [2nCn/(n+1)]*n!

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Deletion in BST

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Minimum Spanning Trees

Spanning Trees
A spanning tree of a connected and undirect graph on n vertices is a subset of n-1 edges that form a
tree.

Number of spanning trees in Graphs:

1. Complete Graph Kn = n*(n^-2)

2. Cycle Graph Cn= n
3. If graph is already a tree = 1
4. Complete Bipartite Graph Km,n =

Kirchoff’s theorem:

If G(V,E) is a graph on n vertices with V = {v1,...,vn} then its graph Laplacian L is an n ⇥ n matrix
whose entries are

The number NT of spanning trees contained in G is given by the following computation:

1. Choose a vertex vj and eliminate the j-th row and column from L to get a new matrix L’.
2. Determinant of new matrix L’ will be equal to the number of spanning trees.
Nt = Det(L’)

Example-

Solution:

if we choose vj = v1 then remove 1st row and col

determinant = 2(3x1 – 1) +1(-1 -0) = 4 -1 =3 spanning tress.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Rooted Spanning Tree:

A rooted spanning tree of a directed graph is rooted tree containing edges of the graph such that
every vertex of the graph is an endpoint of one of the edges in tree.
Note:-
In a connected directed graph G in which each vertex has same in-degree and out-degree has a
rooted spanning tree.

Growing a minimum spanning tree

Assume that we have a connected, undirected graph G = (V, E) with a weight function w: E →R and
we wish to find a minimum spanning tree for G.

Properties:-
1. Let (u, v) be a minimum-weight edge in a connected graph G, then (u, v) belongs to some
minimum spanning tree of G.
2. If an edge (u, v) is contained in some minimum spanning tree, then it is a light edge crossing
some cut of the graph.
3. Let e be a maximum-weight edge on some cycle of connected graph G = (V, E), There is a
minimum spanning tree that doesn’t include e.
4. A graph has a unique minimum spanning tree if, for every cut of the graph, there is a unique
light edge crossing the cut.
5. If all edge weights of a graph are positive, then any subset of edges that connects all vertices
and has minimum total weight must be a minimum spanning tree.
6. Let T be a minimum spanning tree of a graph G, and let L be the sorted list of the edge weights
of T. For any other minimum spanning tree T’ of G, the list L is also the sorted list of edge
weights of T’.
7. Given a graph G and a minimum spanning tree T , suppose that we decrease the weight of one
of the edges in T by some positive integer k, it will still remain M.S.T for G.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Kruskal’s algorithm (Greedy Algorithm) O(Elogv)

In Kruskal’s algorithm, the set A is a forest whose vertices are all those of the given graph. The safe
edge added to A is always a least-weight edge in the graph that connects two distinct components.

It uses a disjoint-set data structure to maintain several disjoint sets of elements. Each set contains
the vertices in one tree of the current forest.

We can determine whether two vertices u and v belong to the same tree by testing whether
FIND-SET(u) equals FIND-SET(v). To combine trees, Kruskal’s algorithm calls the UNION procedure.

1. Lines 1-3 Initialize the set A the empty set and create |V| trees, one containing each vertex.
2. Line 4, edges are sorted in ascending order O(ElogE)
3. The for loop in lines 5–8 examines edges in order of weight, from lowest to highest. The loop
checks, for each edge (u, v), whether the endpoints u and v belong to the same tree. If they do,
then the edge (u, v) cannot be added to the forest without creating a cycle, and the edge is
discarded.
4. If u and v don’t belong to same set that edge (u, v) will be added to A.
5. Line 8 merges the vertices in the two trees.

Properties:-
1. Kruskal’s algorithm can return different spanning trees for the same input graph G, depending
on how it breaks ties when the edges are sorted into order. When graph has duplicate weight
edges.
2.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Time Complexity Analysis of Kruskal’s algorithm:

The running time of Kruskal’s algorithm for a graph G(V, E) depends on how we implement the
disjoint-set data structure.

1. Initializing the set A in line 1 takes O(1) time

2. The time to sort the edges in line 4 is O(ElogE).
3. Line 2-3, there are V, MAKE-SET operations.
4. The for loop of lines 5–8 performs O(E) FIND-SET and UNION operations on the disjoint-set
forest.
5. These, line 2-3 and 5-8, take a total of time.
6.
7. Total running time of kruskal’s algorithm is O(ElogE), As , we have logE = Logv

Running time of Kruskal’s algorithm is O(ElogV)

Note - A is Minimum Spanning Tree for graph G

Prim’s algorithm (Greedy Algorithm) O(Elogv)

Prim’s algorithm operates much like Dijkstra’s algorithm for finding shortest paths in a graph. Prim’s
algorithm has the property that the edges in the set A always form a single tree. The tree starts
from an arbitrary root vertex r and grows until the tree spans all the vertices in V. Each step adds to
the tree A, a light edge that connects A to an isolated vertex—one on which no edge of A is incident.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

During execution of the algorithm, all vertices that are not in the tree reside in a min-priority queue
Q based on a key attribute.
For each vertex v, the attribute v.key is the minimum weight of any edge connecting v to a vertex in
the tree;

1.
2.

1. Lines 1–5 set the key of each vertex to (except for the root r, whose key is set to 0 so that it
will be the first vertex processed), set the parent of each vertex to NIL, and initialize the min-
priority queue Q to contain all the vertices.
2. Line 6, until Q is empty
3. Line 7, extract minimum from min-heap( priority queue)
4. The for loop of lines 8–11 updates the key and attributes of every vertex v adjacent to u but
not in the tree.

The running time of Prim’s algorithm depends on how we implement the min-priority queue Q.
If we implement Q as a binary min-heap

1. We can use the BUILD-MIN-HEAP procedure to perform lines 1–5 in O(V) time.
2. The body of the while loop executes |V| times.
3. Since each EXTRACT-MIN operation takes O(logv) time, the total time for all calls to
EXTRACT-MIN is O(vlogv).
4. The for loop in lines 8–11 executes O(E) times altogether, since the sum of the lengths of all
adjacency lists is 2|E|.
5. Within the for loop, we can implement the test for membership in Q in line 9 in constant
time by keeping a bit for each vertex that tells whether or not it is in Q, and updating the bit
when the vertex is removed from Q.
6. The assignment in line 11 involves an implicit DECREASE-KEY operation on the min-heap,
which a binary min-heap supports in O(log V) time.

Total Time Complexity: O(vlogv + Elogv) = o(Elogv), same as of Kruskal’s algorithm.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Improvements: -

1. If we use Fibonacci heap to hold |V| elements, EXTRACT_MIN takes O(logv) but
DECREASE_KEAY takes O(1) hence T.C will be O( vlogv + E) = O(E)
2. If we use Binomial heap to hold |V| elements, EXTRACT_MIN and DECREASE_KEY take
O(logn) hence T.C will be O(V + E)logv

Prim’s Algorithm with Adjacency Matrix is O(V^2).

Que 23.2-4 Suppose that all edge weights in a graph are integers in the range from 1 to |V |.
How fast can you make Kruskal’s algorithm run? What if the edge weights are integers in the range
from 1 to W for some constant W?
Solution:
In general If we knew that all of
the edge weights in the graph were integers in the range from 1 to |V|, then we could sort the
edges in O(V + E) time using counting sort. There are E edges and range of integer is up to V. Since
the graph is connected, V = O(E), and so the sorting time is reduced to O(E). This would yield a total
running time of =

If the edge weights were integers in the range from 1 to W for some constant W. then we can use
counting sort O( E + W) = O(E) since w is a constant. Same as first part

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Prim’s Algorithm

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Prim’s Algorithm

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Que 23.2-5
Suppose that all edge weights in a graph are integers in the range from 1 to |V|. How fast can you
make Prim’s algorithm run? What if the edge weights are integers in the range from 1 to W for
some constant W?
Solution
The time taken by Prim’s algorithm is determined by the speed of the queue operations (
Decrease_Key and Extract_Min). With the queue implemented as a Fibonacci heap, it takes
O(logv + E) time.
We can improve the running time of Prim’s algorithm if W is a constant by implementing the queue
as an array Q[0 . .W + 1] (using the W + 1 slot for key = )
Then EXTRACT-MIN takes only O(W) = O(1) and DECREASE-KEY takes only O(1) time. This gives a
total running time of O(E).

However, if the range of edge weights is 1 to |V|, then EXTRACT-MIN takes with this data
structure. So the total time spent doing EXTRACT-MIN

Que 23.2-7
Suppose that a graph G has a minimum spanning tree already computed. How quickly can we
update the minimum spanning tree if we add a new vertex and incident edges to G?
Solution:
We can compute a minimum spanning tree of G in O(VlogV) time using Prim’s algorithm with
Fibonacci-heap.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Single-Source Shortest Paths

In a shortest-paths problem (Greedy Algorithm), we are given a weighted, directed graph G(V, E),
with weight function w: E →R mapping edges to real-valued weights. The weight w(p) of path p =
{v0, v1, …… vk} is the sum of the weights of its constituent edges.

We define the shortest-path weight from u to v by

A shortest path from vertex u to vertex v is then defined as any path p with weight

1. Single-source shortest-paths problem: given a graph G(V, E), we want to find a shortest path
from a given source vertex to each vertex .

2. Single-destination shortest-paths problem: find a shortest path to a given destination vertex t

from each vertex v . By reversing the direction of each edge in the graph, we can reduce this
problem to a single-source problem.

3. Single-pair shortest-path problem: Find a shortest path from u to v for given vertices u and v. If
we solve the single-source problem with source vertex u, we solve this problem also.

4. All-pairs shortest-paths problem: find a shortest path from u to v for every pair of vertices u and
v. Although we can solve this problem by running a single source algorithm once from each
vertex, we usually can solve it faster.

Optimal substructure of a shortest path: a shortest path between two vertices contains other
shortest paths within it. Sub-paths of shortest paths are shortest paths.

Negative-weight edges If the graph G(V, E) contains no negative weight cycles reachable from the
source s, then for all , the shortest-path weight remains well defined, even if it has
a negative value.
If the graph contains a negative-weight cycle reachable from s, however, shortest-path weights are
not well defined. No path from s to a vertex on the cycle can be a shortest path, we can always find
a path with lower weight by following the proposed “shortest” path and then traversing the
negative-weight cycle. If there is a negative-weight cycle on some path from s to v,
we define

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Note:-
1. Dijkstra’s algorithm, assume that all edge weights in the input graph are nonnegative.
2. Bellman-Ford algorithm, allow negative-weight edges in the input graph and produce a correct
answer as long as no negative-weight cycles are reachable from the source. If there is such a
negative-weight cycle, the algorithm can detect and report its existence.

Relaxation
For each vertex , we maintain an attribute v.d, which is an upper bound on the weight of
a shortest path from source s to v. We call v.d a shortest-path estimate.

We initialize the shortest-path estimates and predecessors by the following Θ(V) time procedure.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

The process of relaxing an edge (u, v) consists of testing whether we can improve the shortest path
to v found so far by going through u and, if so updating

Following code performs a relaxation step on edge (u, v) in O(1) time:

Dijkstra’s algorithm
Dijkstra’s algorithm solves the single-source shortest-paths problem on a weighted, directed graph
G(V, E) for the case in which all edge weights are nonnegative. We assume that

Dijkstra’s algorithm maintains a set S of vertices whose final shortest-path weights from the source
s have already been determined. The algorithm repeatedly selects the vertex with the
minimum shortest-path estimate, adds u to S, and relaxes all edges leaving u. We use a min-priority
queue Q of vertices, keyed by their d values.

1. Line 1 initializes the d and values in the usual way.

2. Line 2 initializes the set S to the empty set.
3. Line 3 initializes the min-priority queue Q to contain all the vertices in V.
4. While loop of lines 4–8, line 5 extracts a vertex u from Q = V- S and line 6 adds it to set S.
5. Lines 7–8 relax each edge (u, v) leaving u, thus updating the estimate v.d and the predecessor
if we can improve the shortest path to v found so far by going through u.

While loop of lines 4–8 iterates exactly |V| times. The running time of Dijkstra’s algorithm depends
on how we implement the min-priority queue. Since the total number of edges in all the adjacency
lists is |E|, 7-8 lines for loop iterates a total of |E| times, and thus the algorithm calls DECREASE-KEY
at most |E| times overall.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

(a) The situation just before the first iteration of the while loop of lines 4–8. The shaded vertex has
the minimum d value and is chosen as vertex u in line 5. (b)–(f) The situation after each successive
iteration of the while loop. The shaded vertex in each part is chosen as vertex u in line 5 of the next
iteration. The d values and predecessors shown in part (f) are the final values.

1. Consider first the case in which we maintain the min-priority queue by taking advantage of the
vertices being numbered 1 to |V|. We simply store v.d in the vth entry of an array.
a. Each INSERT and DECREASE-KEY operation takes O(1) time.
b. Each EXTRACT-MIN operation takes O(V) time ( since we have to search the entire array).
c. Hence total time will be

2. If we us Binary Heap to maintain min priority queue.

a. Each EXTRACT-MIN operation then takes time O(logv) As before, there are |V| such
operations. The time to build the binary min-heap is O(V).
b. Each DECREASE-KEY operation takes time O(logV), and there are still at most |E| such
operations.
c. The total running time is therefore O(vlogv + Elogv) which is O(Elogv) if all vertices are
reachable from the source.

3. If we use Fibonacci Heap, EXTRACT_MIN operation takes O(logv) and DECREASE_KEY operation
takes O(1), total TC will be O(vlogv +E)

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

The Bellman-Ford algorithm O(VE)

The Bellman-Ford algorithm solves the single-source shortest-paths problem in the general case in
which edge weights may be negative.
Given a weighted, directed graph G(V, E) with source s and weight function w: E→ R, the Bellman-
Ford algorithm returns a Boolean value indicating whether or not there is a negative-weight cycle
that is reachable from the source. If there is such a cycle, the algorithm indicates that no solution
exists. If there is no such cycle, the algorithm produces the shortest paths and their weights.

The algorithm relaxes edges, progressively decreasing an estimate v.d on the weight of a shortest
path from the source s to each vertex until it achieves the actual shortest-path weight
.

1. Line 1 initializes the d and for all vertices.

2. Line 2, for loop iterates |V| - 1 times. Each pass is one iteration of the for loop of lines 2–4 and
consists of relaxing each edge of the graph once.
3. After making |V| - 1 passes, lines 5–8 check for a negative-weight cycle and return the
appropriate Boolean value.

The Bellman-Ford algorithm runs in the O(VE), since the initialization in line 1 takes Θ(V) time, each
of the |V|- 1 passes over the edges in lines 2–4 takes Θ(E) time, and the for loop of lines 5–7 takes
O(E) time.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Prim’s algorithm

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Single-source shortest paths in directed acyclic graphs Θ(V + E)

By relaxing the edges of a weighted dag (directed acyclic graph) G = (V, E) according to a topological
sort of its vertices, we can compute shortest paths from a single source in Θ(V + E) time.
Shortest paths are always well defined in a dag, since even if there are negative-weight edges, no
negative-weight cycles can exist.

All-Pairs Shortest Paths

We can solve an all-pairs shortest-paths problem by running a single-source shortest-paths
algorithm |V| times, once for each vertex as the source.

1. If all edge weights are nonnegative, we can use Dijkstra’s algorithm.

a. If we use the linear-array implementation of the min-priority queue, the running time is

b. The binary min-heap implementation of the min-priority queue yields a running time of

c. The min-priority queue with a Fibonacci heap, yielding a running time of

2. If the graph has negative-weight edges, we cannot use Dijkstra’s algorithm

We must run the slower Bellman-Ford algorithm once from each vertex. The resulting running
time is which on a dense graph is

Floyd-Warshall algorithm Θ(V^3)

It’s dynamic programming algorithm, which runs in time .’

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Greedy Technique
Algorithms for optimization problems typically go through a sequence of steps, with a set of choices
at each step.
A greedy algorithm always makes the choice that looks best at the moment. That is, it makes a
locally optimal choice in the hope that this choice will lead to a globally optimal solution.

Note: Greedy algorithms do not always yield optimal solutions, but for many problems they do.

Optimal substructure
A problem exhibits optimal substructure if an optimal solution to the problem contains within it
optimal solutions to subproblems.

Fractional Knapsack: O(nlogn)

Objective: Object need to be taken to maximize profit without damage to knapsack

N = no. of objects
M = Capacity of Object
Wi = Weight of ith object
Pi = Profit of ith object

We first compute the value per pound pi/wi for each item. Obeying a greedy strategy, the thief
begins by taking as much as possible of the item with the greatest value per pound.
Thus, by sorting the items by value per pound, the greedy algorithm runs in O(nlogn) time.

Algorithm: Greedy fractional knapsack gives optimal solution by giving priority to both profit and
weight.

1. Take a array to store Profit/Weight info

for(i = 1 to n)
job[i] = Pi/Wi
2. Sort the array in descending order O(nlogn)
3. Take one by one object until the capacity of knapsack becomes zero O(n)

Que 16.2-6 Show how to solve the fractional knapsack problem in O(n) time.
Solution
Use a linear-time median algorithm to calculate the median m of the pi/wi ratios. Next, partition the
items into three sets: G = {i : pi/wi > m}, E = {i : pi/wi = m}, and L = {i : pi/wi < m}; this step takes
linear time.
The running time is given by the recurrence whose solution is T(n)=O(n).

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Huffman codes O(nlgon)

Suppose we have a 100,000-character data file that we wish to store compactly. Only 6 different
characters appear.

If we use a fixed-length code, we need 3 bits to represent 6 characters: a = 000, b = 001. . . f = 101.
This method requires 300,000 bits to code the entire file. Can we do better?

A variable-length code can do considerably better than a fixed-length code, by giving frequent
characters short codewords and infrequent characters long codewords.

(a) The tree corresponding to the fixed-length code a = 000. . . f = 101.

(b) The tree corresponding to the optimal prefix code a = 0, b = 101. . . f = 1100.

To analyze the running time of Huffman’s algorithm, we assume that Q is implemented as a binary
min-heap.
1. For a set C of n characters, we can initialize Q in line 2 in O(n) time using the BUILD-MIN-HEAP
procedure.
2. Repeatedly extract the two nodes and replace them in the queue and replace them in the queue
with a new node z with their merger. The for loop in lines 3–8 executes exactly n - 1 times.
3. Each loop iteration require O(logn)
4. Thus, loop contributes O(nlogn)

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Optimal Merge Pattern: O(nlogn)

There are given n files with different number of records. Merge all files into one file optimally with
minimum number of records movement.
1. Every time take 2 minimum and merge them

T.C Is O(nlogn)

Example Minimum number of records movement required to merge five files.

A-10 B-20 C-15 D-5 E-25

Total movements required: 15 + 30 + 45 + 75 = 165

Job Sequencing with Deadline: O(n^2)

1. Single CPU at a time, single job can be executed only.

2. In a span of time more than one job can be executed in round – robin fashion. 5 jobs are
executing one by one
3. Interchange is not allowed (R.R not allowed)
4. Arrival time of all jobs are equal
5. 1 – unit of time of running time for every job eg: one month compulsory neither less or nor
more

Eg: consider the following job sequencing with deadline problem

Solution {J2, J1} is possible but not {J1, J2}, find the optimal solution that gives maximum profit.

Max allowed month is 2, take an array of two slots, Month 2->[ J1] take maximum profit job that can
be completed in 2nd month, among all take maximum job which can be completed either in first
month or 2nd month. That is J4
|J4|J1| total profit = 80+ 55 = 135

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Algorithm:

1. Sort all the jobs in descending order based on profit. O(nlogn)

2. Find max deadline and take array of that size O(n) to find max dead line
3. For every slot i, apply linear search to find a job which contains deadline >= i. this step will
take O(n^2) time because we have sorted on profit but searching based on deadline. Hence
binary search can’t be applied
4. As we find first job, stop and delete the job.

Jobs are sorted in descending order:

Max deadline is 5, so, take an array of size 5.

i=5, we start searching from LHS, first job is J1 whose deadline >=5, copy it and delete it from.

i=4, J5 is the first job whose deadline >= 4, copy it

1 2 3 4 5
J5 J1
i=3
1 2 3 4 5
J3 J5 J1
i=2
1 2 3 4 5
J6 J3 J5 J1
i=1
1 2 3 4 5
J2 J6 J3 J5 J1

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Dynamic Programming

Dynamic programming, like the divide-and-conquer method, solves problems by combining the
solutions to subproblems.
We typically apply dynamic programming to optimization problems. Such problems can have many
possible solutions. Each solution has a value, and we wish to find a solution with the optimal
(minimum or maximum) value.

1. D.P is applicable when the subproblems are not independent. Subproblems share subproblems.
2. DAC algorithm does more work than necessary repeatedly solving the common subproblems.
3. D.P solves each problem exactly once and save answer to a table.

Two methods of storing the result in memory

1. Memorization ( Top – Down)
Whenever we solve a problem first time we store it and reuse it next time.
2. Tabulation (Bottom – Up)
We pre-compute the solutions in linear fashion and later use them.

Dynamic Programming is used when

1. Overlapping Subproblems
2. Optimal Substructure
Eg: Floyd warshall, Bell-man ford etc

Note – Longest path problem doesn’t have optimal substructure property.

Matrix-chain multiplication

We are given a sequence (chain) <A1, A2, ………, An> f n matrices to be multiplied, and we wish to
compute the product
A1A2 … An
If the chain of matrices is <A1, A2, A3, A4>then we can fully parenthesize the product in 5 distinct
ways:

We state the matrix-chain multiplication problem as follows: given a chain

Fully parenthesize the product A1A2 . . . An in a way that minimizes the number of
scalar multiplications.
Counting the number of parenthesizations

Denote the number of alternative parenthesizations of a sequence of n matrices by P(n). When n = 1

we have just one matrix and therefore only one way to fully parenthesize the matrix product.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

When n >= 2, a fully parenthesized matrix product is the product of two fully parenthesized matrix
subproducts, and the split between the two subproducts may occur between the kth and (k+1)st
matrices for any k = 1, 2, 3, ….. n-1 Thus, we obtain the recurrence:

Solution to the recurrence relation is .

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

A recursive solution

To determine the minimum cost of parenthesizing

be the minimum number of scalar
Multiplications needed to compute the matrix

m[i , j] if i=j then it’s a trivial problem.

Note: i <=k < j

n(n+1)/2 = 10 distinct function calls

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

n(n+1)/2 distinct function calls, and each function call needs O(n). overall time complexity will be
O(n^3).

Example - A2x5 B5x4 C4x2 D2x3

Answer is 68 multiplications are required to multiply 4 matrices A,B,C and D

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

0/1 knapsack problem

• n items.
• Item i is worth $vi, weighs wi pounds.
• Find a most valuable subset of items with total weight ≤ W.
• Have to either take an item or not take it, but can’t part of it.

Recursive Relation:

Recursive Program: O(2^n)

int knapSack(int W, int wt[], int val[], int n)
{

if (n == 0 || W == 0)
return 0;
if (wt[n-1] > W)
return knapSack(W, wt, val, n-1);

else return max( val[n-1] + knapSack(W-wt[n-1], wt, val, n-1),

knapSack(W, wt, val, n-1)
);
}

Time Complexity: it will be a complete binary tree. O(2^n)

Using Dynamic Programming O(m*n)

There will be distinct function calls (m+1)*(n+1).

The Time complexity using Dynamic Programming will be O(m*n), Where m is the capacity of
knapsack and n is the number of items.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Longest Common Subsequence:

Given two sequences X and Y, we say that a sequence Z is a common subsequence of X and Y if Z is a
subsequence of both X and Y.
1. For example, if X = <A, B, C, B, D, A, B> and Y = <B, D, C, A, B, A> the sequence <B, C, A> is a
common subsequence of both X and Y.
2. The sequence <B, C, A> is not a longest common subsequence (LCS) of X and Y, it has length 3.
3. The sequence <B, C, B, A>, which is also common to both X and Y, has length 4.
4. The sequence <B, C, B, A> is an Longest Common Subsequence of X and Y, since X and Y have no
common subsequence of length 5 or greater.

Length common subsequence of X and Y.

Brute Force Approach:

We would enumerate all subsequences of X and check each subsequence to see whether it is also a
subsequence of Y, keeping track of the longest subsequence we find. X has 2^m subsequences hence
This approach requires O(n*2^m) time complexity.

A recursive solution

Worst case time complexity of this recurrence relation will be o(2^(m+n)

Using Dynamic Programming

There are total distinct (m+1)(n+1) function calls, hence time complexity will be O(m*n) and Space
complexity will be O(m*n) to store dynamic programming table.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

When the value matches take the value from diagonal up cell and increment it by 1.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Fibonacci Series:-

Recursive Program:

Let T(n) denote the running time of FIB(n). Since FIB(n) contains two recursive calls plus a constant
amount of extra work, we obtain the recurrence

I. Time complexity for this recurrence relation is O(2^n).

II. Space Complexity is O(n)

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Fibonacci Series using Dynamic Programming:

In dynamic programming we will solve every problem only once. There are 6 unique function calls.

Time complexity = O(n)

Space Complexity = O(n)

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Multistage Graph
A multistage graph G = (V, E) is directed graph in which vertices are partitioned in K – disjoint sets
where k>=2
In addition, if (u, v) is an edge E then u ∈ Vi and v ∈ Vi+1.
Let C(i, j) be the cost of edge (i, j), the cost of a path from S to T is the sum of the costs of the edge
on the path.
The multistage graph problem is to find the min cost path from S to T.
Total distinct function calls are equal to number of edges, hence time complexity will be O(E)

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Sum of Subset Problem

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Representations of graphs

We can choose between two standard ways to represent a graph G = (V, E) as a collection of
adjacency lists or as an adjacency matrix.
Either way applies to both directed and undirected graphs.
Because the adjacency-list representation provides a compact way to represent sparse graphs those
for which |E| is much less than |V^2|.
We may prefer an adjacency-matrix representation, however, when the graph is dense |E| is close
to |V^2| or or when we need to be able to tell quickly if there is an edge connecting two given
vertices.

Adjacency list representations of a graph G = (V, E) consists of an array Adj of , one for each
vertex in V. For each the adjacency list Adj[u] contains all the vertices v such that there is an
edge That is, Adj[u] consists of all the vertices adjacent to u in G.

1. If G is a directed graph, the sum of the lengths of all the adjacency lists is |E|
2. If G is a undirected graph, the sum of the lengths of all the adjacency lists is 2*|E|
3. The adjacency-list requires the amount of memory Θ(V + E)
4. For weighted graph, we store the weight w(u, v) of the edge with vertex v in u’s
adjacency list.
5. No quicker way to determine whether a given edge (u, v) is present in the graph than to search
for v in the adjacency list Adj[u].

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Adjacency-matrix representation of a graph G(V, E), we assume that the vertices are numbered 1, 2,
. . . .|V| in some arbitrary manner. Then the adjacency-matrix representation of a graph G consists
of a |V|x|V| matrix.
A = Aij such that

1. The adjacency matrix of a graph requires Θ(V^2) memory, independent of the number of edges
in the graph.
2. The adjacency matrix A of an undirected graph is its own transpose:
3. For weighted graph, we store w(u, v) of the edge as the entry in row u and column v
of the adjacency matrix, otherwise nil if row doesn’t exist.
4. Adjacency matrices carry a further advantage for unweighted graphs: they require only one bit
per entry.

Que 22.1-1
Given an adjacency-list representation of a directed graph, how long does it take to compute the
out-degree of every vertex? How long does it take to compute the in-degrees?
Solution
Out-degree of each vertex
1. Graph out-degree of a vertex u is equal to the length of Adj[u].
2. The sum of the lengths of all the adjacency lists in Adj is |E|.
3. Thus the time to compute the out-degree of every vertex is Θ(V + E)

In-degree of each vertex

1. The in-degree of a vertex u is equal to the number of times it appears in all the lists in Adj.
2. If we search all the lists for each vertex, time to compute the in-degree of every vertex is Θ(VE)
3. Alternatively, we can allocate an array T of size |V| and initialize its entries to zero.
4. We only need to scan the lists in Adj once, incrementing T[u] when we see u in the lists.
5. The values in T will be the in-degrees of every vertex.
6. This can be done in Θ(V + E) time with Θ(V) additional storage.

Que:-
Given an adjacency-matrix representation of a directed graph, how long does it take to compute the
out-degree of every vertex? How long does it take to compute the in-degrees?

The adjacency-matrix A of any graph has Θ(V^2) entries, regardless of the number of edges in the
graph.
For a directed graph, computing the out-degree of a vertex u is equivalent to scanning the row
corresponding to u in A and summing the 1’s.
Computing the out-degree of every vertex is equivalent to scanning all entries of A. hence, Θ(V^2)
Computing the in-degree of a vertex u is equivalent to scanning the column corresponding to u in A
and summing the 1’s. Thus it requires Θ(V^2)

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Que 22.1-3

For computing GT from G, for both the adjacency list and adjacency-matrix representations of G.
What will be time complexity respectively?
Solution
For the adjacency matrix representation
1. To compute the graph transpose, we just take the matrix transpose.
2. Every entry above the diagonal, and swapping it with the entry that occurs below the diagonal.
3. This takes O(V^2).

For adjacency list:

We will maintain an initially empty adjacency list representation of the transpose. Then, we scan
through every list in the original graph. If we are in the list corresponding to vertex v and see u as an
entry in the list, then we add an entry of v to the list in the transpose graph corresponding to vertex
u. Since this only requires a scan through all of the lists, it only takes time O(|E| + |V|).

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Breadth-first search
Given a graph G = (V, E) and a distinguished source vertex s, breadth-first search systematically
explores the edges of G to “discover” every vertex that is reachable from s.
1. It computes the distance (smallest number of edges) from s to each reachable vertex.
2. It also produces a “breadth-first tree” with root s that contains all reachable vertices. For
any vertex v reachable from s.
3. The simple path in the breadth-first tree from s to v corresponds to a “shortest path” from s
to v in G, that is, a path containing the smallest number of edges.
4. The algorithm works on both directed and undirected graphs.
5. The algorithm discovers all vertices at distance k from s before discovering any vertices at
distance k + 1.

1. To keep track of progress, breadth-first search colors each vertex white, gray, or black.
2. All vertices start out white and may later become gray and then black.
3. If and vertex u is black, then vertex v is either gray or black; that is, all vertices
adjacent to black vertices have been discovered.
4. Gray vertices may have some adjacent white vertices.

The breadth-first-search procedure BFS below assumes that the input graph G = (V, E) is represented
using adjacency lists.

The procedure BFS works as follows:

1. Except source vertex s, lines 1–4 paint every vertex white, set u.d to be infinity for each
vertex u, and set the parent of every vertex to be NIL
2. Line 5 paints s gray, since we consider it to be discovered as the procedure begins.
3. Line 6 initializes s.d to 0, and line 7 sets the predecessor of the source to be NIL.
4. Lines 8–9 initialize Q to the queue containing just the vertex s.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

5. The while loop of lines 10–18 iterates as long as there remain gray vertices, which are
discovered vertices that have not yet had their adjacency lists fully examined.

Analysis:

1. After initialization, breadth-first search never whitens a vertex, and thus the test in line 13
ensures that each vertex is enqueued at most once, and hence dequeued at most once.
2. The operations of enqueuing and dequeuing take O(1) time, and so the total time devoted
to queue operations is O(V).
3. Because the procedure scans the adjacency list of each vertex only when the vertex is
dequeued, it scans each adjacency list at most once.
4. Since the sum of the lengths of all the adjacency lists is |E|, the total time spent in scanning
adjacency lists is O(E).
5. The overhead for initialization is O(V).
6. Thus the total running time of the BFS procedure is O(V + E) with adjacency-list
representation of G.
7. If adjacency Matrix is used, BFS T.C will be O(V^2), for each vertex u we dequeue we'll have
to examine all vertices v to decide whether or not v is adjacent to u.

Shortest Path
BFS finds the distance to each reachable vertex in a graph G = (V, E) from a given source vertex
. Define the shortest-path distance from s to v as the minimum number of edges in
any path from vertex s to vertex v; if there is no path from s to v, then

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Lemma 22.1
Let G = (V, E) be a directed or undirected graph, and let be an arbitrary vertex. Then, for any
edge

Lemma 22.2
Let G = (V, E) be a directed or undirected graph, and suppose that BFS is run on G from a given
source vertex Then upon termination, for each vertex

BFS Applications

1. If graph is connected.
2. If graph is cyclic.
3. Single shortest path if unweighted graph or weight is same for all edges.
4. To check Bipartite Graph property.
5. To know number of connected components.
6. No. of levels in the graph

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Depth-first search

“To search “deeper” in the graph whenever possible”

Depth-first search explores edges out of the most recently discovered vertex v that still has
unexplored edges leaving it.
Once all of v’s edges have been explored, the search “backtracks” to explore edges leaving the
vertex from which v was discovered. This process continues until we have discovered all the vertices
that are reachable from the original source vertex.

If any undiscovered vertices remain, then depth-first search selects one of them as a new source,
and it repeats the search from that source. The algorithm repeats this entire process until it has
discovered every vertex.
Unlike breadth-first search, whose predecessor subgraph forms a tree, the predecessor subgraph
produced by a depth-first search may be composed of several trees, because the search may repeat
from multiple sources.

The predecessor subgraph of a depth-first search forms a depth-first forest comprising several
depth-first trees.

Each vertex is initially white, is grayed when it is discovered in the search, and is blackened when it
is finished, that is, when its adjacency list has been examined completely. This technique
guarantees that each vertex ends up in exactly one depth-first tree, so that these trees are disjoint.

Besides creating a depth-first forest, depth-first search also timestamps each vertex. Each vertex v
has two timestamps: the first timestamp v.d records when v is first discovered (and grayed), and the
second timestamp v.f records when the search finishes examining v’s adjacency list (and blackens v).

The procedure DFS below records when it discovers vertex u in the attribute u.d and when it
finishes vertex u in the attribute u.f . These timestamps are integers between 1 and 2|V|, since
there is one discovery event and one finishing event for each of the |V| vertices.
For every vertex u, u.d < u.f

Note: -
1. Vertex u is WHITE before time u.d, GRAY between time u.d and time u.f, and BLACK
thereafter.
2. The input graph G may be undirected or directed.
3. The variable time is a global variable that we use for timestamping.

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Analysis
What is the running time of DFS? The loops on lines 1–3 and lines 5–7 of DFS take time Θ(V).
The procedure DFS-VISIT is called exactly once for each vertex . Since the vertex u on which
DFS-VISIT is invoked must be white and the first thing DFS-VISIT does is paint vertex u gray.
During an execution of DFS-VISIT G(V, E), the loop on lines 4–7 executes |Adj[v]| times.
Since

The total cost of executing lines 4–7 of DFS-VISIT is Θ(E). Running time of DFS is therefore Θ(V+E)

Manu Thakur (www.facebook.com/worstguymanu)

 Algorithms

Depth First Search Tree

Classification of edges

Tree edges are those edges (u, v) is a tree edge if v was first discovered by exploring edge (u, v).

Back edges are those edges (u, v) connecting a vertex u to an ancestor v in a depth-first tree. We
consider self-loops, which may occur in directed graphs, to be back edges.

Forward edges are those nontree edges (u, v) connecting a vertex u to a descendant v in a depth-
first tree.
Cross edges they can go between vertices in the same depth-first tree, as long as one vertex is not
an ancestor of the other, or they can go between vertices in different depth-first trees.
Note: - In a depth-first search of an undirected graph G, every edge of G is either a tree edge or a
back edge.

Applications of DFS:
1. Graph is connected.
2. Finding no. of connected components.
3. To check if graph contains cycle.
4. Finding no. of articulation point in O(E) time.
5. Finding no. of Bridges in O(E) time.
6. Topological sort

Manu Thakur (www.facebook.com/worstguymanu)