Design and Analysis of Algorithms

Algorithms and Computations Complexity

Lecture 3
References:
Introduction to Algorithms, Thomas H. Cormen, 2ed edition,

Instructor: Prashant mishra

BIAS, Bhimtal
 What are the Algorithms?

Algorithm:

 Is any well-defined computational procedure that takes some

value, or set of values, as input and produces some value, or set
of values, as output.

 An algorithm is thus a sequence of computational steps that

transform the input into output.

 It is a tool for solving a well-specified computational problem.

 They are written in a pseudo code which can be implemented in

the language of programmer’s choice.
In addition all algorithm must follow Following criteria:
Input: Zero or more quantities are externally
supplied .
Output: At least one quantity is produced.
certainty: Each instruction must be clear and
unambiguous.
Finiteness: An algorithm must terminates out
after a finite number of steps.
Effectiveness: Every instruction must be very
basic so that it can be carried out.
 Example: sorting numbers.

Input: A sequence of n numbers {a3 , a1, a2,...,an }

Output: A reordered sequence of the input

{a1 , a2, a3,...,an } such that a1≤a2 ≤a3… ≤an .

Input instance: {5, 2, 4, 1, 6, 3}.

Output : {1, 2, 3, 4, 5, 6}.

An instance of a problem consists of the input (satisfying

whatever constraints are imposed in the problem statement)
needed to compute a solution to the problem.
 Example: sorting numbers.

Sorting is a fundamental operation.

Many algorithms existed for that purpose.

The best algorithm to use depends on:

The number of items to be sorted.
possible restrictions on the item values
kind of storage device to be used: main memory, disks, or tapes.
 Correct and incorrect algorithms
• Algorithm is correct if, for every input instance, it
ends with the correct output. We say that a correct
algorithm solves the given computational problem.

• An incorrect algorithm might not end at all on some

input instances, or it might end with an answer other
than the desired one.

• We shall be concerned only with correct algorithms.

 Practical Examples

Internet and Networks.

The need to access large amount of information with the
shortest time.
Problems of finding the best routs for the data to travel.
Algorithms for searching this large amount of data to
quickly find the pages on which particular information
resides.

Electronic Commerce.
The ability of keeping the information (credit card
numbers, passwords, bank statements) private, safe, and
secure.
Algorithms involves encryption/decryption techniques.
 Hard problems
We can identify the Efficiency of an algorithm
from its speed (how long does the algorithm
take to produce the result).

Some problems have unknown efficient

solution.

These problems are called NP-complete

problems.
 Hard problems
Nobody has proven that an efficient algorithm
for one cannot exist.

Several NP-complete problems are similar, but not

identical.

If we can show that the problem is NP-complete, we

can spend our time developing an efficient algorithm
that gives a good, but not the best possible solution.

Example of NP-complete problem: “traveling

salesman”.
 Why do we need the algorithms?
One always need to proof that his solution method terminate
and does the correct answer.

Computers may be fast but they are not infinitely fast, and
memory may be cheap but it is not free. This resources should
be used wisely.

As an example:

There is two algorithms for sorting; merge sort algorithm, and

insertion sort algorithm

Insertion sort takes an execution time equal c1*n2 to sort n

items.
Merge sort takes an execution time equal c2*n log2n to sort n
items.
Why do we need algorithms?- cont.

c1 and c2 are constants.

Insertion sort usually has a smaller constant factor than merge

sort, so that c1< c2

Two computers; computer A running insertion sort, and

computer B running merge sort.

Assuming Computer A executes one billion instructions per

second and computer B executes only ten million instructions
per second, so that computer A is 100 times faster than
computer B.
 Why do we need algorithms?- cont.

• Suppose (c1 =2, and c2 =50).

• To sort one million number, computer A takes,

2 x (106 ) 2 instruction /109 instruction/sec = 2000 sec.

• While computer B takes, 50 x 106 x log2 106 instruction /107

instruction/sec = 100 sec.
 Getting started

We will use the insertion sort algorithm to:

Define the pseudo code we are use.
we analyze its running time.

Introduce the divide-and conquer approach to the

design of algorithms and use it to develop an
algorithm called merge sort.

We end with an analysis of the merge sort running

time.
 Insertion sort

Input: A sequence of n numbers {a3 , a1,

a2,...,an }
Output: A reordered sequence of the input {a1
, a2, a3,...,an } such that a1≤a2 ≤a3… ≤an
 Insertion Sort
Sorted array/list is by inserting one item at a
time
– Simple to implement
– Efficient on small data sets
– Efficient on already almost ordered data sets
 Insertion Sort
• Start with a sequence of size 1
• Repeatedly insert remaining elements
 Insertion sort – pseudo code.
• The insertion algorithm

• This algorithm takes as parameter an array of the sequence

numbers to be sorted A[1 … n], the number of elements in the
array is denoted by length[ A].
• When this algorithm finish, the input array will contains the
sorted sequence numbers (sort in place: the sorted items
(output) occupy the same storage as the original ones (input) ).
 Insertion Sort
void InsertionSort(double a[],int n){
for (int j = 2; j <= n; j++){
double temp = a[j];
int i;
for (i = j - 1; i > 0 && temp < a[i]; i--){
a[ i + 1] = a[ i];
}
a[ i + 1] = temp;
}
}
 The insertion sort
We will use A={5, 2, 4, 6, 1, 3} as an input instance.
The index j indicate the element to be inserted.
A[1…j-1] contains the sorted elements.
A[j+1…n] contains the elements still to be sorted.
 Pseudo-code conventions
Algorithms are typically written in pseudo-code that is
similar to C/C++ and JAVA.

Pseudo-code differs from real code with:

It is not typically concerned with issues of software engineering.
Issues of data abstraction, and error handling are often ignored.

Indentation indicates block structure. For example, the

body of the for loop that begins on line 1 consists of lines
2-8, and the body of the while loop that begins on line 5
contains lines 6-7 but not line 8.

The symbol "▹" indicates that the remainder of the line is

a comment.
Pseudo-code conventions, cont…

A multiple assignment of the form i ← j ← e assigns to

both variables i and j the value of expression e; it should
be treated as equivalent to the assignment j ← e
followed by the assignment i ← j.

Variables ( such as i, j, and key) are local to the given

procedure. We shall not us global variables without
explicit indication.

Array elements are accessed by specifying the array

name followed by the index in square brackets. For
example, A[i] indicates the ith element of the array A.
The notation “…" is used to indicate a range of values
within an array. Thus, A[1…j] indicates the sub-array of
A consisting of the j elements A[1], A[2], . . . , A[j].
Pseudo-code conventions, cont…

A particular attributes is accessed using the attributes

name followed by the name of its object in square
brackets.
For example, we treat an array as an object with the
attribute length indicating how many elements it
contains( length[ A]).
 Analyzing algorithms

We need to predict the resources( memory,

communication bandwidth, or computer hardware) that
the algorithm requires.

Most often it is computational time that we want to

measure.

By analyzing several candidate algorithms for a

problem, the most efficient one can be easily identified.

Unless other thing specified, we shall assume that our

algorithms will be implemented on a generic one
processor computer with the instructions being executed
one after another, with no concurrent operations.
 Analysis of algorithms
What’s more important than performance?
• modularity
• correctness
• maintainability
• functionality
• robustness
• user-friendliness
• programmer time
• simplicity
• extensibility
• reliability
The Problem-solving Process

Analysis
Problem
specification
Design

Algorithm

Implementation

Program

Compilation

Executable
(solution)
 Components of an Algorithm
• Variables and values
• Instructions
• Procedures…. functions
• Selections…. If statement
• Repetitions…. For , while loops
• Documentation
 Running time & Input size
The running time of an algorithm on a particular input is
the number of primitive operations or “steps”
executed.
The best notation for “input size” depends on the
problem being studied. For example:
In sorting or computing discrete Fourier transforms the
most natural measure is the no. of item in the input
For example the array size n for sorting.
On the other hand in multiplying two integers the best
measure is the total no. of bits needed to represent the
input in ordinary binary notation .
• The running time depends on the input: an
already sorted sequence is easier to sort.
• Parameterize the running time by the size of the
input, since short sequences are easier to sort
than long ones.
• Generally, we seek upper bounds on the running
time, because everybody likes a guarantee.
 Complexity of Algorithm
The complexity of an algorithm M is the function f(n)
which give the running time and/or storage space
requirement of the algorithm in term of the size n of
input data. Frequently storage space required by an
algorithm is simply a multiple of data size n.
Accordingly, unless or otherwise stated the term
“complexity” will refer to running time of an
algorithm.
Cases for complexity function

Worst case :The maximum value of f(n) for

any possible input.
Average case: the expected value of f(n).
Best case: Sometimes we consider minimum
possible value of f(n) called the best case.
 Asymptotic notation
 Theta notation(Θ)
 Big oh notation(O)
 Small oh notation(o)
 Omega notation (Ω)
 Little omega notation (ω)
 Theta notation (Θ)

For a given function g(n), we denote

Θ(g(n))={f(n): there exist positive constant
C1,C2 and n0 such that
0 ≤ C1g(n) ≤ f(n) ≤ C2g(n)for all n≥ n0
f(n) = Θ(g(n))

C2(g(n))

f(n)

C1(g(n))

n0 n
 Big oh notation(O)

The theta –notation asymptotically bounds a function

from above and below. When we have only an
asymptotically upper bound ,
We use O-notation.
For a given function g(n), we denote by Og(n) the set
of functions
Og(n)= {f(n): there exist positive constant C and n0 such
that
0 ≤ f(n) ≤ Cg(n) for all n≥ n0
 f(n) = O(g(n))

f(n) = O(g(n))

C(g(n))

f(n)

n
n0
 Omega notation(Ω)

Just as O-notation provides an asymptotic upper

bound on a function, Ω notation provide an
asymptotic lower bound
For a given function g(n), we denote by Ω(g(n))
the set of functions
Ω(g(n))={f(n): there exist positive constant C
and n0 such that
0 ≤ Cg(n) ≤ f(n) for all n≥ n0 }
 f(n)=Ω(g(n))
f(n)=Ω(g(n))

f(n)

C(g(n))

no n
 Small oh notation(o)

The asymptotic upper bound provided by O-notation

may or may not be asymptotically tight, but the
bound o-notation to denote an upper bound that is
not asymptotically tight.
We formally define o(g(n)) as the set

Og(n)= {f(n): for any constant c>0 there exist

a constant C >0and n0 >0 such that
0 ≤ f(n) < Cg(n) for all n≥ n0 }
 Little omega notation (ω)
By analogy ω- notation is to Ω-notation is same as o
notation to O- notation.
We use ω- notation to denote a lower bound that is not
asymptotically tight .
It is defined by :
f(n) Є ω(g(n)) iff g(n) Є o(f(n))
Formally we defined ω(g(n)) as the set
ω(g(n))={ f(n) : for any positive constant c>0, there
exists a constant n0 >0 such that
0 ≤ c(g(n)) < f(n) for all n≥n0 }
Comparison of function
 Reflexivity
f(n) = Θ(f(n))

f(n) = O(f(n))

f(n) = Ω(f(n))
 Symmetry

f(n) = Θ(g(n)) iff g(n) = Θ(f(n))

 Transitivity
f(n) = Θ(g(n)) and g(n) = Θ(h(n)) imply f(n)= Θ(h(n))

f(n) = O(g(n)) and g(n) = O(h(n)) imply f(n)= O(h(n))

f(n) = Ω (g(n)) and g(n) = Ω(h(n)) imply f(n)= Ω(h(n))

f(n) = o(g(n)) and g(n) = o(h(n)) imply f(n)= o(h(n))

f(n) = ω(g(n)) and g(n) = ω(h(n)) imply f(n)= ω(h(n))

 Transpose symmetry
f(n) =O(g(n)) iff g(n) = Ω(f(n))

f(n) =o(g(n)) iff g(n) = ω(f(n))

 Analysis of insertion sort
The time taken by insertion sort depends on the input (sorting
thousand items takes more time than sorting three items).

In general, the time taken by an algorithm grows with the size

of the input, so it is traditional to describe the running time of
a program as a function of the size of its input.

Input size depends on the problem being studied

( Ex: sort algorithm: the number of items to be sorted.
Multiplying two integers: the total number of bits needed to
represent the input in ordinary binary notation.
In graphs: the input size can be described by the numbers of
vertices and edges in the graph).
 Analysis of insertion sort

The running time of an algorithm on a particular

input is the number of primitive operations or "steps"
executed.

A constant amount of time is required to execute each

line of our pseudo-code. One line may take a different
amount of time than another line, but we shall assume
that each execution of the ith line takes time ci ,
where ci is a constant.
 Analysis of insertion sort – cont.

We start by presenting the INSERTION-SORT procedure with

the time "cost" of each statement and the number of times each
statement is executed.

For each j = 2, 3, . . . , n, where n = length[ A], we let i be the

number of times the while loop test in line 5 is executed for
that value of j. When a for or while loop exits in the usual way
(i.e., due to the test in the loop header), the test is executed one
time more than the loop body.

We assume that comments are not executable statements, and

so they take no time.
Analysis of insertion sort
 Analysis of insertion sort – cont.

The running time of the algorithm is the sum of running times

for each statement executed; a statement that takes ci steps to
execute and is executed n times will contribute ci n to the total
running time.

To compute T( n), the running time of INSERTIONSORT, we

sum the products of the cost and times columns, obtaining
 Analysis of insertion sort – cont.

For the INSERTION-SORT, the best case occurs if the array

is already sorted. For each j = 2, 3, . . . , n, we then find that
A[i] ≤ key in line 5 when i has its initial value of j - 1.
Thus tj= 1 for j = 2, 3, … , n, and the best-case running time
is:
(tj: is the number of time the while loop is executed for that value of j)
T(n) =c1n+c2(n−1)+c4(n−1)+c5(n−1)+c8(n−1).
=(c1 +c2 +c4+c5 +c8)n −(c2 +c4 +c5 +c8).

This running time can be expressed as an + b for constants a

and b that depend on the statement costs ci ; it is thus a linear
function of n.
 Analysis of insertion sort – cont.

If the array is in reverse sorted order-that is, in decreasing order-the worst

case results. We must compare each element A[ j] with each element in the
entire sorted sub-array A[1…j - 1], and so t j = j for j = 2, 3,…, n. Noting
that

This will result

This worst-case running time can be expressed as an 2 + bn + c for

constants a, b, and c, that again depend on the statement costs ci ; it
is thus a quadratic function of n.
Worst-case and average-case analysis
The worst-case running time of an algorithm is an upper
bound on the running time for any input. Knowing it gives us
a guarantee that the algorithm will never take any longer.

The "average case" is often roughly as bad as the worst case.

Suppose that we randomly choose n numbers and apply
insertion sort. How long does it take to determine where in
sub-array A[1… j -1]to insert element A[ j], On average, half
the elements in A[1…j - 1] are less than A[ j], and half the
elements are greater.
On average, therefore, we check half of the sub-array A[1…j -
1], so t j = j/2. If we work out the resulting average-case
running time, it turns out to be a quadratic function of the
input size, just like the worst-case running time.
 Insertion sort – Order of growth.

This worst-case running time can be expressed as an 2 + bn + c

for constants a, b, and c

To find the order of growth we consider only the leading term

of a formula (e.g, an2 ), since the lower-order terms are
relatively insignificant for large n.

We also ignore the leading term's constant coefficient, since

constant factors are less significant than the rate of growth in
determining computational efficiency for large inputs.
 Insertion sort – Order of growth.

Thus we write that insertion sort for example, has a

worst-case running time of Θ(an 2) (pronounced
"theta of n-squared").

We usually consider one algorithm to be more

efficient than another if its worst-case running time
has a lower order of growth.

Due to constant factors and lower-order terms, this

evaluation may be in error for small inputs. But not
for large enough inputs
 Designing algorithms
There are many ways to design an algorithm.

Insertion sort uses an incremental approach: having sorted

the sub-array A[1…j - 1], we insert the single element A[ j]
into its proper place, yielding the sorted sub-array A[1…j].

Another approach to design is the divide-and-conquer

approach which has a recursive structure to solve a given
problem; they break the problem into several sub-problems
that are similar to the original problem but smaller in size,
solve the sub-problems recursively, and then combine these
solutions to create a solution to the original problem.
 The divide-and-conquer approach
It involves three steps:
Divide the problem into a number of sub-problems.

Conquer the sub-problems by solving them recursively. If the sub-

problem sizes are small enough, however, just solve the sub-problems
in a straightforward manner.

Combine the solutions to the sub-problems into the solution for the
original problem.

Example: merge sort.

Divide: Divide the n element sequence to be sorted into two n-
subsequences of n/2 elements each.
Conquer: Sort the two subsequences recursively using merge sort.
Combine: Merge the two sorted subsequences to produce the sorted
answer.
Design and Analysis of Algorithms
Divide and conquer approach
Merge sort

References:

Introduction to Algorithms, Thomas H. Cormen, 2ed edition

Lecture 4

Instructor : Prashant Mishra

BIAS,Bhimtal
 Divide and conquer approach
The divide and conquer paradigm involve three steps at each
level of recursion:
Divide: divide the problem into a number of sub problem .
Conquer: conquer the sub problems by solving them
recursively. if the sub problem size is small enough
,however just solve the sub problem in straightforward
manner.
Combine: combine the solution of the sub problems into the
solution for original problem.

the merge sort algorithm follow the divide and conquer

paradigm.
 The recursion “bottom out” when the sequence to be sorted has length
1, in which case there is no work to be done ,since every sequence of
length 1 is already in sorted order.
 The key operation of the merge sort algorithm is the merging, we use an
auxiliary procedure MERGE (A,p,q,r) where A is an array such that
p<=q<r.
 The procedure assume that the sub arrays A[p…q] andA[q+1..r] in
sorted order .It merges them to form a single sorted sub array that
replace the current sub array A [p ..r].
 Our MERGE procedure takes Θ(n), where n = r-p+1 is the number of
elements being merged.
 Merge sort
The procedure MERGE-SORT(A, p, r) sorts the elements in the sub-array
A[ p…r].

The divide step simply computes an index q that partitions A[ p…r] into two
sub-arrays: A[ p…q], containing n/2 elements, and A[ q + 1…r], containing
n/2 elements.

To sort the entire sequence A ={A[1], A[2], . . . , A[ n]}, we make the initial
call MERGE-SORT( A, 1, length[ A]), where length[ A] = n.
 Merge sort – cont.
The operation of merge sort on the array A = {1, 5, 2, 4, 6, 3, 2, 6}.
 Merge sort

The key operation of the merge sort algorithm is the merging

of two sorted sequences in the "combine" step. To perform the
merging, we use an auxiliary procedure MERGE(A, p, q, r),
where A is an array and p, q, and r are indices numbering
elements of the array such that p ≤ q < r.

The procedure assumes that the sub-arrays A[ p…q] and A[ q

+ 1…r] are in sorted order. It merges them to form a single
sorted sub-array that replaces the current sub-array A[ p…r].
 Merge algorithm – cont.

The operation of lines 10-17 in the call MERGE(A, 9, 12, 16).

 Merge algorithm – cont.
The operation of lines 10-17 in the call MERGE(A, 9, 12, 16)
 Merge sort

Our MERGE procedure takes time Θ( n), where n = r - p + 1 is

the number of elements being merged, and it works as follows.
Suppose we have two arrays of sorted numbers, with the
smallest number is the first in each array. We wish to merge
the two arrays into a single sorted output array. Our basic step
consists of choosing the smaller of the first two numbers on
both arrays, removing it from its array , and placing this
number onto the output array. We repeat this step until one
input array is empty, at which time we just take the remaining
input array and place it onto the output array. Computationally,
each basic step takes constant time, since we are checking
just two first numbers. Since we perform at most n basic steps,
merging takes Θ( n) time.
 Review: Analysis of Merge Sort
Statement Effort

MergeSort(A, left, right) {

if (left < right) { Θ(1)
mid = floor((left + right) / 2); Θ(1)
MergeSort(A, left, mid); T(n/2)
MergeSort(A, mid+1, right); T(n/2)
Merge(A, left, mid, right); Θ(n)
}
}
 Analysis of merge sort - cont.
To simplify the analysis we assume that the original problem size is a power
of 2. Each divide step then yields two subsequences of size exactly n/2.

Merge sort on just one element takes constant time.

When we have n > 1 elements:

Divide: The divide step just computes the middle of the sub-array, which takes
constant time. Thus, D( n) = Θ(1).
Conquer: We recursively solve two sub-problems, each of size n/2, which
contributes 2T (n/2) to the running time.
Combine: The MERGE procedure on an n-element sub-array takes time Θ( n),
so C( n) = Θ( n).
When we add the functions D( n) and C( n) for the merge sort analysis, we
are adding a function that is Θ( n) and a function that is Θ(1). This sum is a
linear function of n that is Θ( n) . Adding it to the 2T (n/2) term from the
"conquer" step gives the recurrence for the worst-case running time T (n)
of merge sort:
 Analysis of Merge Sort
To simplify the analysis we assume that the original problem
size is a power of 2
Assume n=2k for k>=1
T(n) = 2 T(n/2) + bn + c
T(n/2) = 2T((n/2) /2) + b(n/2) + c
= 2[2T(n/4) + b(n/2) + c] + bn +c
= 4 T(n/4)+ bn +2c +bn +c
=4 T(n/4) + 2bn+ (1 + 2) c = 22 T(n/22)+2bn+(20+21)
= 4 [2T((n/4)/2) + b(n/4) + c] +2bn + (1+2)c
=8 T(n/8) + 3bn+ (1+2+4)c
=23 T(n/23) + 3bn+ (20+21+22)c

=2k T(n/2k) +kbn+(20+21+…+2k-1)c

T(1) = a, since n=2k log n = log2k = k
T(n) = 2k. a + k bn + (20+21+…+2k-1) c
= b. n log n + a n+ c n
= O (n log2 n) Worst case
 Recursion tree

Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

 Recursion tree

Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

Recursion tree
Algorithms and Computations Complexity
Quick sort
Lecture 5
References:
Introduction to Algorithms, Thomas H. Cormen, 2ed edition,

Instructor: Prashant mishra

BIAS,Bhimtal
 Quick sort
• Quick sort, like merge sort, is based on the divide-and-conquer
paradigm introduced in previous section. Here is the three-step
divide-and-conquer process for sorting typical sub array
A[p.…..r].
• Divide: Partition (rearrange) the array A[p……r] into two
(possibly empty) sub arrays A[p…...q – 1] and A[q + 1…...r]
such that each element of A[p…...q – 1] it less than or equal to
A[q], which is, in turn, less than or equal to each element of
A[q + 1..r]. Compute the index q as part of this partitioning
procedure.
• Conquer: Sort the two sub arrays A [p ..q – 1] and A[q + 1..r]
by recursive calls to quick sort.
• Combine: Since the sub arrays are sorted in place, no work is
needed to combine them: the entire array A[p…...r] is now
sorted.
QUICKSORT (A, p, r)
1. if p < r
2. then q ← Partition (A, p, r)
3. Quick sort (A, p, q – 1)
4. Quick sort (A, q + 1, r)

To sort an entire array A, the initial call is

QUICKSORT (A, 1, length [A]).
 Partitioning the Array

PARTITION (A, p, r)
1. x ← A [r]
2. i ← p – 1
3. for j ← p to r – 1
4. do if A[ j] ≤ x
5. then i ← i + 1
6. exchange A [i] ↔ A [j]
7. exchange A [i + 1] ↔ A [r]
8. return i + 1
Figure below shows the operation of
PARTITION on an 8-element array.
PARTITION always selects an element x = A
[r] as a pivot element around which to partition
the sub array A[p…...r].
Partitioning the array
i p, j r

(a) 2 8 7 1 3 5 6 4

p, i j r

(b) 2 8 7 1 3 5 6 4

p. i j r

(c) 2 8 7 1 3 5 6 4

p, i j r

(d) 2 8 7 1 3 5 6 4

p i j r

(e) 2 1 7 8 3 5 6 4

p i j r

(f) 2 1 3 8 7 5 6 4

p i j r

(g) 2 1 3 8 7 5 6 4

p i j r

(h) 2 1 3 8 7 5 6 4

p i r

(i) 2 1 3 4 7 5 6 8

The Operation of PARTITION on a Sample Array

 Worst-case of quick sort

• Input sorted or reverse sorted.

• Partition around min or max element.

• One side of partition always has no elements.

T(n) = T(0) + T(n–1) + cn
 Best case partitioning
In the most even possible split PARTITION
produce two sub problems each of size no
more than n/2 since one is of size n/2 and
one of size n/2 -1 in this case quick sort run
much faster .
The recurrence for running time is then
T(n)<= 2T(n/2) + Θ (n)
Which by case 2 of master theorem has the
solution T(n) = O(nlgn)
 Balanced partitioning
The average case running time of quicksort is
much closer to the best case than to the worst
case .
Suppose for ex. That the PARETITION
algorithm always produce a 9:1 proportional
split .
We obtain the recurrence
T(n) <=T(9n/10) +T(n/10)+cn
The total cost of quick sort is O(nlgn)
 Design and Analysis of Algorithms
RecurrenceAndMasterTheorem
Lecture 5
References:
Introduction to Algorithms, Thomas H. Cormen, 2ed edition,

Instructor: Prashant mishra

BIAS,Bhimtal
 Recurrence
 A recurrence is an equation or inequality that
describe a function in term of its value on smaller
inputs.
 For example the worst case running time T(n) of the
merge sort procedure can be described by the
recurrence

T(n)= {Θ(1) if n=1

{ aT(n/b) +D(n) +C(n) if n>1

Whose solution was claimed to be T(n)=Θ(n lg n)
 Recurrences

• MERGE-SORT
– Contains details:
• T(n) = Θ(1) if n=1
T(n/2)+ T(n/2)+ Θ(n) if n>1
• Ignore details, T(n) = 2T(n/2)+ Θ(n).
– T(n) = Θ(1) if n=1
2T(n/2)+ Θ(n) if n>1

3
 Methods for solving recurrence
Substitution method
Recursion method
The master method
 The Substitution Method
• Two steps:
1. Guess the form of the solution.
• By experience, and creativity.
• By some heuristics.
– If a recurrence is similar to one you have seen before.
» T(n)=2T(n/2+17)+n, similar to T(n)=2T(n/2)+n, , guess O(nlg n).
– Prove loose upper and lower bounds on the recurrence and then reduce the
range of uncertainty.
» For T(n)=2T(n/2)+n, prove lower bound T(n)= Ω(n), and prove upper
bound T(n)= O(n2), then guess the tight bound is T(n)=O(nlg n).
• By recursion tree.
2. Use mathematical induction to find the constants and show
that the solution works.

5
 Solve T(n)=2T(n/2)+n
• Guess the solution: T(n)=O(nlg n),
– i.e., T(n)≤ cnlg n for some c.
• Prove the solution by induction:
– Suppose this bound holds for n/2, i.e.,
• T(n/2)≤ cn/2 lg (n/2).
– T(n) ≤ 2(cn/2 lg (n/2))+n
• ≤ cn lg (n/2))+n
• = cn lg n - cn lg 2 +n
• = cn lg n - cn +n
• ≤ cn lg n (as long as c≥1)

Question: Is the above proof complete? Why?

6
 Boundary (base) Condition
• In fact, T(n) =1 if n=1, i.e., T(1)=1.
• However, cnlg n =c×1×lg 1 = 0, which is odd with
T(1)=1.
• Take advantage of asymptotic notation: it is
required T(n)≤ cnlg n hold for n≥ n0 where n0 is a
constant of our choosing.
• Select n0 =2, thus, n=2 and n=3 as our induction
bases. It turns out any c ≥ 2 suffices for base cases
of n=2 and n=3 to hold.

7
 Subtleties
• Guess is correct, but induction proof not work.
• Problem is that inductive assumption not strong
enough.
• Solution: revise the guess by subtracting a lower-order
term.
• Example: T(n)=T(n/2)+T(n/2)+1.
– Guess T(n)=O(n), i.e., T(n) ≤ cn for some c.
– However, T(n) ≤c n/2+c n/2+1 =cn+1, which does not
imply T(n) ≤ cn for any c.
– Attempting T(n)=O(n2) will work, but overkill.
– New guess T(n) ≤ cn – b will work as long as b ≥ 1.
– (Prove it in an exact way).
8
 Avoiding Pitfall
• It is easy to guess T(n)=O(n) (i.e., T(n) ≤
cn) for T(n)=2T(n/2)+n.
• And wrongly prove:
– T(n) ≤ 2(c n/2)+n
• ≤ cn+n
• =O(n).  wrongly !!!!
• Problem is that it does not prove the exact
form of T(n) ≤ cn.
9
 Find bound, ceiling, floor, lower term–
domain transformation
• Find the bound: T(n)=2T(n/2)+n (O(nlogn)
• How about T(n)=2T(n/2)+n ?
• How about T(n)=2T(n/2)+n ?
– T(n)≤2T(n/2+1)+n
– Domain transformation
• Set S(n)=T(n+a) and assume S(n)=2S(n/2)+n (so S(n)=O(nlogn))
• S(n)=2S(n/2)+n T(n+a)=2T(n/2+a)+n
• T(n)≤2T(n/2+1)+n T(n+a) ≤ 2T((n+a)/2+1)+n
• Thus, set n/2+a=(n+a)/2+1, get a=2.
• so T(n)=S(n-2)=O((n-2)log(n-2)) = O(nlogn).
• How about T(n)=2T(n/2+19)+n ?
– Set S(n)=T(n+a) and get a=38.
• As a result, ceiling, floor, and lower terms will not affect.
– Moreover, the master theorem also provides proof for this.

10
 Changing Variables
• Suppose T(n)=2T(√n)+lg n.
• Rename m=lg n. so T(2m)=2T(2m/2)+m.
• Domain transformation:
– S(m)=T(2m), so S(m)=2S(m/2)+m.
– Which is similar to T(n)=2T(n/2)+n.
– So the solution is S(m)=O(m lg m).
– Changing back to T(n) from S(m), the solution is T(n)
=T(2m)=S(m)=O(m lg m)=O(lg n lg lg n).

11
 The Recursion-tree Method
• Idea:
– Each node represents the cost of a single subproblem.
– Sum up the costs with each level to get level cost.
– Sum up all the level costs to get total cost.
• Particularly suitable for divide-and-conquer
recurrence.
• Best used to generate a good guess, tolerating
“sloppiness”.
• If trying carefully to draw the recursion-tree and
compute cost, then used as direct proof.
12
 Recursion Tree for T(n)=3T(n/4)+Θ(n2)
T(n) cn2 cn2
T(n/4) T(n/4) T(n/4) c(n/4)2 c(n/4)2 c(n/4)2

T(n/16) T(n/16) T(n/16) T(n/16) T(n/16) T(n/16) T(n/16) T(n/16) T(n/16)

(a) (b) (c)
cn2 cn2

c(n/4)2 c(n/4)2 (3/16)cn2

c(n/4)2

log 4n (3/16)2cn2
c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2

T(1)T(1)T(1) T(1)T(1)T(1) Θ(nlog 43)

3log4n= nlog 43 Total O(n2)
(d) 13
 Solution to T(n)=3T(n/4)+Θ(n2)
• The height is log 4n,
• #leaf nodes = 3log 4n= nlog 43. Leaf node cost: T(1).
• Total cost T(n)=cn2+(3/16) cn2+(3/16)2 cn2+
⋅⋅⋅ +(3/16)log 4n-1 cn2+ Θ(nlog 43)
=(1+3/16+(3/16)2+ ⋅⋅⋅ +(3/16)log 4n-1) cn2 + Θ(nlog 43)
<(1+3/16+(3/16)2+ ⋅⋅⋅ +(3/16)m+ ⋅⋅⋅) cn2 + Θ(nlog 43)
=(1/(1-3/16)) cn2 + Θ(nlog 43)
=16/13cn2 + Θ(nlog 43)
=O(n2).

14
 Prove the above Guess
• T(n)=3T(n/4)+Θ(n2) =O(n2).
• Show T(n) ≤dn2 for some d.
• T(n) ≤3(d (n/4)2) +cn2
≤3(d (n/4)2) +cn2
=3/16(dn2) +cn2
≤ dn2, as long as d≥(16/13)c.

15
 One more example
• T(n)=T(n/3)+ T(2n/3)+O(n).
• Construct its recursive tree (Figure 4.2,
page 71).
• T(n)=O(cnlg3/2n) = O(nlg n).
• Prove T(n) ≤ dnlg n.

16
Recursion Tree of T(n)=T(n/3)+ T(2n/3)+O(n)

17
 Master Method/Theorem
• Theorem 4.1 (page 73)
– for T(n) = aT(n/b)+f(n), n/b may be n/b or n/b.
– where a ≥ 1, b>1 are positive integers, f(n) be a non-
negative function.
1. If f(n)=O(nlogba-ε) for some ε>0, then T(n)= Θ(nlogba).
2. If f(n)= Θ(nlogba), then T(n)= Θ(nlogba lg n).
3. If f(n)=Ω(nlogba+ε) for some ε>0, and if af(n/b) ≤cf(n)
for some c<1 and all sufficiently large n, then T(n)=
Θ(f(n)).

18
 Implications of Master Theorem
• Comparison between f(n) and nlogba (<,=,>)
• Must be asymptotically smaller (or larger) by a
polynomial, i.e., nε for some ε>0.
• In case 3, the “regularity” must be satisfied, i.e.,
af(n/b) ≤cf(n) for some c<1 .
• There are gaps
– between 1 and 2: f(n) is smaller than nlogba, but not
polynomially smaller.
– between 2 and 3: f(n) is larger than nlogba, but not
polynomially larger.
– in case 3, if the “regularity” fails to hold.
19
 Application of Master Theorem
• T(n) = 9T(n/3)+n;
– a=9,b=3, f(n) =n
– nlogba = nlog39 = Θ (n2)
– f(n)=O(nlog39-ε) for ε=1
– By case 1, T(n) =Θ (n2).
• T(n) = T(2n/3)+1
– a=1,b=3/2, f(n) =1
– nlogba = nlog3/21 = Θ (n0) = Θ (1)
– By case 2, T(n)= Θ(lg n).
20
 Application of Master Theorem
• T(n) = 3T(n/4)+nlg n;
– a=3,b=4, f(n) =nlg n
– nlogba = nlog43 = Θ (n0.793)
– f(n)= Ω(nlog43+ε) for ε≈0.2
– Moreover, for large n, the “regularity” holds for
c=3/4.
• af(n/b) =3(n/4)lg (n/4) ≤ (3/4)nlg n = cf(n)
– By case 3, T(n) =Θ (f(n))=Θ (nlg n).
21
 Exception to Master Theorem
• T(n) = 2T(n/2)+nlg n;
– a=2,b=2, f(n) =nlg n
– nlogba = nlog22 = Θ (n)
– f(n) is asymptotically larger than nlogba , but not
polynomially larger because
– f(n)/nlogba = lg n, which is asymptotically less
than nε for any ε>0.
– Therefore,this is a gap between 2 and 3.
22
 Where Are the Gaps
f(n), case 3, at least polynomially larger
nε Gap between case 3 and 2
c1
nlogba f(n), case 2: within constant distances
c2
nε Gap between case 1 and 2
f(n), case 1, at least polynomially smaller

Note: 1. for case 3, the regularity also must hold.

2. if f(n) is lg n smaller, then fall in gap in 1 and 2
3. if f(n) is lg n larger, then fall in gap in 3 and 2
4. if f(n)=Θ(nlogbalgkn), then T(n)=Θ(nlogbalgk+1n). (as exercise)
23
 Proof of Master Theorem
• The proof for the exact powers, n=bk for k≥1.
• Lemma 4.2
– for T(n) = Θ(1) if n=1
– aT(n/b)+f(n) if n=bk for k≥1
– where a ≥ 1, b>1, f(n) be a nonnegative function,
– Then logbn-1
– T(n) = Θ(nlogba)+ ajf(n/bj)
j=0

• Proof:
– By iterating the recurrence
– By recursion tree (See figure 4.3)

24
Recursion tree for T(n)=aT(n/b)+f(n)

25
 Proof of Master Theorem (cont.)
• Lemma 4.3:
– Let a ≥ 1, b>1, f(n) be a nonnegative function defined on
exact power of b, then
logbn-1
– g(n)=  ajf(n/bj) can be bounded for exact power of b as:
j=0
1. If f(n)=O(nlogba-ε) for some ε>0, then g(n)= O(nlogba).
2. If f(n)= Θ(nlogba), then g(n)= Θ(nlogba lg n).
3. If f(n)= Ω(nlogba+ε) for some ε>0 and if af(n/b) ≤cf(n) for
some c<1 and all sufficiently large n ≥b, then g(n)= Θ(f(n)).

26
 Proof of Lemma 4.3
• For case 1: f(n)=O(nlogba-ε) implies f(n/bj)=O((n /bj)logba-ε), so
logbn-1 logbn-1
• g(n)=  ajf(n/bj) =O(  a j(n /bj)logba-ε )
j=0 j=0
logbn-1 logbn-1
• = O(nlogba-ε  a j/(blogba-ε)j ) = O(nlogba-ε
 a j/(aj(b-ε)j))
j=0 j=0
logbn-1
• = O(nlogba-ε  (bε)j ) = O(nlogba-ε (((bε ) logbn-1)/(bε-1) )
j=0
• = O(nlogba-ε (((blogbn)ε -1)/(bε-1)))=O(nlogba n-ε (nε -1)/(bε-1))
• = O(nlogba )
27
 Proof of Lemma 4.3(cont.)
• For case 2: f(n)= Θ(nlogba) implies f(n/bj)= Θ((n /bj)logba), so

logbn-1 logbn-1
• g(n)=  ajf(n/bj) = Θ(  aj(n /bj)logba)
j=0 j=0
logbn-1 logbn-1
• = Θ(nlogba  aj/(blogba)j ) = Θ(nlogba  1)
j=0 j=0

• = Θ(nlogba logbn) = Θ(nlogbalg n)

28
 Proof of Lemma 4.3(cont.)
• For case 3:
– Since g(n) contains f(n), g(n) = Ω(f(n))
– Since af(n/b) ≤cf(n), ajf(n/bj) ≤cjf(n) , why???

logbn-1 j logbn-1 j ∞
– g(n)=  a f(n/bj) ≤  c f(n) ≤ f(n)  cj
j=0 j=0 j=0

– =f(n)(1/(1-c)) =O(f(n))
– Thus, g(n)=Θ(f(n))
29
 Proof of Master Theorem (cont.)
• Lemma 4.4:
– for T(n) = Θ(1) if n=1
– aT(n/b)+f(n) if n=bk for k≥1
– where a ≥ 1, b>1, f(n) be a nonnegative function,
1. If f(n)=O(nlogba-ε) for some ε>0, then T(n)= Θ(nlogba).
2. If f(n)= Θ(nlogba), then T(n)= Θ(nlogba lg n).
3. If f(n)=Ω(nlogba+ε) for some ε>0, and if af(n/b) ≤cf(n)
for some c<1 and all sufficiently large n, then T(n)=
Θ(f(n)).

30
 Proof of Lemma 4.4 (cont.)
• Combine Lemma 4.2 and 4.3,
– For case 1:
• T(n)= Θ(nlogba)+O(nlogba)=Θ(nlogba).
– For case 2:
• T(n)= Θ(nlogba)+Θ(nlogba lg n)=Θ(nlogba lg n).
– For case 3:
• T(n)= Θ(nlogba)+Θ(f(n))=Θ(f(n)) because f(n)=
Ω(nlogba+ε).

31
 Floors and Ceilings
• T(n)=aT(n/b)+f(n) and T(n)=aT(n/b)+f(n)
• Want to prove both equal to T(n)=aT(n/b)+f(n)
• Two results:
– Master theorem applied to all integers n.
– Floors and ceilings do not change the result.
• (Note: we proved this by domain transformation too).
• Since n/b≤n/b, and n/b≥ n/b, upper bound for
floors and lower bound for ceiling is held.
• So prove upper bound for ceilings (similar for lower
bound for floors).

32
 Upper bound of proof for T(n)=aT(n/b)+f(n)
• consider sequence n, n/b,  n/b/b,   n/b /b/b,
…
• Let us define nj as follows:
• nj = n if j=0
• = nj-1/b if j>0
• The sequence will be n0, n1, …, nlogbn
• Draw recursion tree:

33
Recursion tree of T(n)=aT(n/b)+f(n)

34
The proof of upper bound for ceiling
logbn -1
– T(n) = Θ(nlogba)+  ajf(nj)
j=0

– Thus similar to Lemma 4.3 and 4.4, the upper

bound is proven.

35
The simple format of master theorem

• T(n)=aT(n/b)+cnk, with a, b, c, k are

positive constants, and a≥1 and b≥2,
O(nlogba), if a>bk.
• T(n) = O(nklogn), if a=bk.
O(nk), if a<bk.

36
 Summary
Recurrences and their bounds
– Substitution
– Recursion tree
– Master theorem.
– Proof of subtleties
– Recurrences that Master theorem does not
apply to.

37
Design and Analysis of Algorithms
Heap sort
Lecture 7
References:
Introduction to Algorithms, Thomas H. Cormen,
2ed edition,

Instructor: Prashant mishra

BIAS,Bhimtal
Heap

The (binary) heap data structure is an array object that can be viewed as a
nearly complete binary tree. Each node of the tree corresponds to an
element of the array that stores the value in the node. The tree is
completely filled on all levels except possibly the lowest, which is
filled from the left up to a point. An array A that represents a heap is an
object with two attributes: length[A], which is the number of elements
in the array, and heap-size[A], the number of elements in the heap stored
within array A.
The root of the tree is A[1], and given the index i of a node, the indices of its
parent PARENT(i), left child LEFT(i), and right
child RIGHT(i) can be computed simply:
Heap types
There are two type of binary heaps
Max heap :in a max heap ,the max heap property
is that for every node i other than root
A[Parent(i)] =>A[i]
Min heap :in a min heap ,the min heap property is
that for every node i other than root
A[Parent(i)] <=A[i]
For the heap sort algorithm we use max heap
Viewing a heap as a tree we define the height of a
node in a heap to be the longest simple
downwards path from the node to a leaf and we
define the height of the tree as the height of its
root .since a heap of n element is based on a
complete binary tree its height is Θ(n)
 Basic procedures
MAX- HEAPIFY procedure which run in
O(lgn ) time

THE BUILD-MAX HEAP procedure which run in

linear time .Produce a max heap from an unordered
input array .

The HEAP SORT procedure which run in O(nlgn )

time, sort an array in place .
When MAX- HEAPIFYis called it is assumed that
the binary tree rooted at left [i] and right [i] are
max heaps , but that A [i] may be smaller than its
children thus violating the max heap property.
The function of MAX- HEAPIFY is to let the value
at A [i ] “float down” in the max heap so that the
sub tree rooted at index i becomes a max heap.
Maintaining the heap property
Building a heap

We can use the procedure MAX- HEAPIFY in a

bottom up manner to convert an array
A[1…n]=length [A] into a max-heap.
The element in the sub array
A[ (n/2 +1)….n] are all leaves of the tree and so
each is a 1 element heap to begin with .
The procedure Build max heap goes through the
remaining nodes of the tree and run MAX-
HEAPIFY on each one.
Build Max–Heap (A)
1 heap size [A] ←length [A]
2for i ← length [A]/2 down to 1
3 do Max-Heapify (A,i)
The heap sort algorithm

HEAP SORT (A)

1 Build Max –Heap (A)
2 for i ←length [A] down to 2
3 Do exchange A[1] A[i]
4 Heap size[A] ←heap-size[A]-1
5 MAX- HEAPIFY(A,1)
The heap sort procedure take time O (n.lgn) since the call
to Build Max –Heap take time O (n) and each of the n-1
calls to Build Max –Heapify takes times O (lg n)
We need to show that this invariant is true prior to the first loop iteration, that
each iteration of the loop maintains the invariant, and that the invariant provides
a useful property to show correctness when the loop terminates.

Initialization: Prior to the first iteration of the loop, i = ⌊n/2⌋. Each node ⌊n/2⌋ +
1, ⌊n/2⌋ + 2, . . . , n is a leaf and is thus the root of a trivial max-heap.

Maintenance: To see that each iteration maintains the loop invariant, observe
that the children of node i are numbered higher than i. By the loop invariant,
therefore, they are both roots of max-heaps. This is precisely the condition
required for the call MAX-HEAPIFY(A, i) to make node i a max-heap root.
Moreover, the MAX-HEAPIFY call preserves the property that nodes i + 1, i + 2,
. . . , n are all roots of max-heaps. Decrementing i in the for loop update
reestablishes the loop invariant for the next iteration.

Termination: At termination, i = 0. By the loop invariant, each node 1, 2, . . . , n

is the root of a max-heap. In particular, node 1 is.

We can compute a simple upper bound on the running time of BUILD-MAX-

HEAP as follows. Each call to MAX-HEAPIFY costs O(lg n) time, and there are
O(n) such calls. Thus, the running time is O(n lg n). This upper bound, though
correct, is not asymptotically tight.
We can derive a tighter bound by observing that the time for MAX-HEAPIFY to run at a
node varies with the height of the node in the tree, and the heights of most nodes are small.
Our tighter analysis relies on the properties that an n-element heap has height ⌊lg n⌋ and at
most ⌈n/2h+1⌉ nodes of any height h
The time required by MAX-HEAPIFY when called on a node of height h is O(h), so we can
express the total cost of BUILD-MAX-HEAP as

The last summation can be evaluated by substituting x = 1/2 in the formula (A.8), which yields

(A.8)

Thus, the running time of BUILD-MAX-HEAP can be bounded as

Hence, we can build a max-heap from an unordered array

in linear time
Algorithms and Computations Complexity
Data Structures for Disjoint Sets
Lecture 14
References:
Introduction to Algorithms, Thomas H. Cormen, 2ed edition,

Instructor: Prashant mishra

BIAS, Bhimtal
Data Structures for Disjoint Sets
Some applications involve grouping n distinct
elements into a collection of disjoint sets. Two
important operations are then finding which set a
given element belongs to and uniting two sets.
This chapter explores methods for maintaining a
data structure that supports these operations.
Disjoint-set operations
A disjoint-set data structure maintains a collection of disjoint
dynamic sets. Each set is identified by a representative, which is
some member of the set. In some applications, it doesn't matter
which member is used as the representative; we only care that if
we ask for the representative of a dynamic set twice without
modifying the set between the requests, we get the same answer
both times.
As in the other dynamic-set implementations we have studied,
each element of a set is represented by an object. Letting x denote
an object, we wish to support the following operations:
MAKE-SET(x) creates a new set whose only member (and thus
representative) is x.Since the sets are disjoint, we require that x
not already be in some other set.
UNION(x, y) unites the dynamic sets that contain x and y, say Sx
and Sy, into a new set that is the union of these two sets. The two
sets are assumed to be disjoint prior to the operation. The
representative of the resulting set is any member of Sx Sy,
although many implementations of UNION specifically choose
the representative of either Sx or Sy as the new representative.
Since we require the sets in the collection to be disjoint, we
"destroy" sets Sx and Sy, removing them from the collection .
FIND-SET(x) returns a pointer to the representative of the (unique)
set containing x.
An application of disjoint-set data structures
One of the many applications of disjoint-set data structures arises
in determining the connected components of an undirected graph
for example ,shows a graph with four connected components.
The procedure CONNECTED-COMPONENTS that follows uses the
disjoint-set operations to compute the connected components of a
graph. Once CONNECTED-COMPONENTS has been run as a
preprocessing step, the procedure SAME-COMPONENT answers
queries about whether two vertices are in the same connected
component. (The set of vertices of a graph G is denoted by V [G],
and the set of edges is denoted by E[G].)
CONNECTED-COMPONENTS(G)
1 for each vertex v V[G]
2 do MAKE-SET(v)
3 for each edge (u, v) E[G]
4 do if FIND-SET(u) ≠ FIND-SET(v)
5 then UNION(u, v)
SAME-COMPONENT(u, v)
1 if FIND-SET(u) = FIND-SET(v)
2 then return TRUE
3 else return FALSE
Algorithms and Computations Complexity
Graphs
Lecture 13
References:
Introduction to Algorithms, Thomas H. Cormen, 2ed edition,

Instructor: Prashant mishra

BIAS, Bhimtal
Graphs
A graph is an ordered pair
G = (V,E).
 V is the set of vertices of G.
 E is the set of edges of G.
 The elements of E are pairs (v,w)
 of vertices.
Representations of graphs
There are two standard ways to represent a graph G = (V, E): as a
collection of adjacency lists or as an adjacency matrix. Either
way is applicable to both directed and undirected graphs.
The adjacency-list representation is usually preferred, because it
provides a compact way to represent sparse graphs-those for
which |E| is much less than |V|2.
The adjacency-list representation of a graph G = (V, E) consists
of an array Adj of |V| lists, one for each vertex in V . For each u
V, the adjacency list Adj[u] contains all the vertices v such that
there is an edge (u, v) E. That is, Adj[u] consists of all the
vertices adjacent to u in G. (Alternatively, it may contain
pointers to these vertices.) The vertices in each adjacency list
are typically stored in an arbitrary order.

Figure 1: Two representations of an undirected graph. (a) An

undirected graph G having five vertices and seven edges. (b) An
adjacency-list representation of G. (c) The adjacency matrix
representation of G.
Figure 2: Two representations of a directed graph. (a) A
directed graph G having six vertices and eight edges. (b) An
adjacency-list representation of G. (c) The adjacency-matrix
representation of G.
Breadth-first search

The breadth-first-search procedure BFS below assumes

that the input graph G = (V, E) is represented using adjacency
lists. It maintains several additional data structures with each
vertex in the graph. The color of each vertex u €V is stored in
the variable color[u], and the predecessor of u is stored in the
variable π[u]. If u has no predecessor (for example, if u = s or
u has not been discovered), then π[u] = NIL. The distance
from the source s to vertex u computed by the algorithm is
stored in d [u]. The algorithm also uses a first-in, first-out
queue Q to manage the set of gray vertices.
Breadth-first search
BFS(G, s)
1 for each vertex u € V [G] - {s}
2 do color[u] ← WHITE
3 d[u] ← ∞
4 π[u] ← NIL
5 color[s] ← GRAY
6 d[s] ← 0
7 π[s] ← NIL
8Q←Ø
9 ENQUEUE(Q, s)
10 while Q ≠ Ø
11 do u ← DEQUEUE(Q)
12 for each v Adj[u]
13 do if color[v] = WHITE
14 then color[v] ← GRAY
15 d[v] ← d[u] + 1
16 π[v] ← u
17 ENQUEUE(Q, v)
18 color[u] ← BLACK
The operation of BFS on an undirected graph. Tree edges are shown
shaded as they are produced by BFS. Within each vertex u is
shown d[u]. The queue Q is shown at the beginning of each
iteration of the while loop of lines 10-18. Vertex distances are
shown next to vertices in the queue. The procedure BFS works as
follows. Lines 1-4 paint every vertex white, set d[u] to be infinity
for each vertex u, and set the parent of every vertex to be NIL.
Line 5 paints the source vertex s gray, since it is considered to be
discovered when the procedure begins. Line 6 initializes d[s] to
0, and line 7 sets the predecessor of the source to be NIL. Lines
8-9 initialize Q to the queue containing just the vertex s. The
while loop of lines 10-18 iterates as long as there remain gray
vertices, which are discovered vertices that have not yet had their
adjacency lists fully examined. This while loop maintains the
following invariant:
Analysis
Before proving the various properties of breadth-first search, we
take on the somewhat easier job of analyzing its running time
on an input graph G = (V, E). We use aggregate analysis, as we
saw in Section 17.1. After initialization, no vertex is ever
whitened, and thus the test in line 13 ensures that each vertex
is enqueued at most once, and hence dequeued at most once.
The operations of enqueuing and dequeuing take O(1) time, so
the total time devoted to queue operations is O(V). Because the
adjacency list of each vertex is scanned only when the vertex
is dequeued, each adjacency list is scanned at most once. Since
the sum of the lengths of all the adjacency lists is Θ(E), the
total time spent in scanning adjacency lists is O(E). The
overhead for initialization is O(V), and thus the total running
time of BFS is O(V + E). Thus, breadth-first search runs in
time linear in the size of the adjacency-list representation of G.
Depth-first search
The strategy followed by depth-first search is, as its name
implies, to search "deeper" in the graph whenever possible. In
depth-first search, edges are explored out of the most recently
discovered vertex v that still has unexplored edges leaving it. When
all of v's edges have been explored, the search "backtracks" to
explore edges leaving the vertex from which v was discovered.
This process continues until we have discovered all the vertices that
are reachable from the original source vertex. If any undiscovered
vertices remain, then one of them is selected as a new source and
the search is repeated from that source. This entire process is
repeated until all vertices are discovered.
DFS(G)
DFS(G)
1 for each vertex u V [G]
2 do color[u] ← WHITE
3 π[u] ← NIL
4 time ← 0
5 for each vertex u V [G]
6 do if color[u] = WHITE
7 then DFS-VISIT(u)
DFS-VISIT(u)
1 color[u] ← GRAY ▹White vertex u has just been discovered.
2 time ← time +1
3 d[u] time
4 for each v Adj[u] ▹Explore edge(u, v).
5 do if color[v] = WHITE
6 then π[v] ← u
7 DFS-VISIT(v)
8 color[u] BLACK ▹ Blacken u; it is finished.
9 f [u] ▹ time ← time +1
Topological sort
A topological sort of a dag G = (V, E) is a linear ordering of all
its vertices such that if G contains an edge (u, v), then u appears
before v in the ordering. (If the graph is not acyclic, then no
linear ordering is possible.) A topological sort of a graph can be
viewed as an ordering of its vertices along a horizontal line so
that all directed edges go from left to right. Topological sorting
is thus different from the usual kind of "sorting”.
The following simple algorithm topologically sorts a dag.
TOPOLOGICAL-SORT(G)
1 call DFS(G) to compute finishing times f[v] for each vertex v
2 as each vertex is finished, insert it onto the front of a linked list
3 return the linked list of vertices