REGULATION 2021 SCHEME DAA-21CS42

Design and Analysis of Algorithms-21CS42

-Module 2

Prepared by

Dr.C.Vinola
Prof.Saranya

Prepared By: Dr.Vinola.C, Prof.Saranya.C SSCE, Anekal

 REGULATION 2021 SCHEME DAA-21CS42

Module-2

Divide and Conquer: General method, Recurrence equation for divide and conquer, solving it using
Master’s theorem. , Divide and Conquer algorithms and complexity Analysis of Finding the maximum
& minimum, Binary search, Merge sort, Quick sort.
Decrease and Conquer Approach: Introduction, Insertion sort, Graph searching algorithms,
Topological Sorting. It’s efficiency analysis.

Divide and Conquer:

A divide and conquer algorithm works by recursively breaking down a problem into two
or more sub-problems of the same (or related) type (divide), until these become simple enough to
be solved directly (conquer).
Divide-and-conquer algorithms work according to the following general plan:
1. A problem is divided into several subproblems of the same type, ideally of about equal size.
2. The subproblems are solved (typically recursively, though sometimes a different algorithm
is employed, especially when subproblems become small enough).
3. If necessary, the solutions to the subproblems are combined to get a solution to the original
problem.

The divide-and-conquer technique as shown in the Figure, which depicts the case of
dividing a problem into two smaller subproblems, then the subproblems solved separately. Finally
solution to the original problem is done by combining the solutions of subproblems.

problem of size n

subproblem 1of size n/2 subproblem 2 of size n/2

solution to subproblem 1 solution to subproblem 2

solution to the original problem

Prepared By: Dr.Vinola.C, Prof.Saranya.C SSCE, Anekal

 REGULATION 2021 SCHEME DAA-21CS42

Recurrence relation

As mentioned above, in the most typical case of divide-and-conquer a problem’s instance of size n is
divided into two instances of size n/2. More generally, an instance of size n can be divided into b
instances of size n/b, with a of them needing to be solved. (Here, a and b are constants; a ≥ 1 and b >
1.) Assuming that size n is a power of b to simplify our analysis, we get the following recurrence for
the running time T (n):

where f (n) is a function that accounts for the time spent on dividing an instance of size n into
instances of size n/b and combining their solutions. (For the sum example above, a = b = 2 and f (n) =
1.) Recurrence is called the general divide-and-conquer recurrence. Obviously, the order of growth of
its solution T (n) depends on the values of the constants a and b and the order of growth of the
function f (n).

Solving it using Master’s theorem:

The efficiency analysis of many divide-and-conquer algorithms is greatly simplified by the following
theorem

Prepared By: Dr.Vinola.C, Prof.Saranya.C SSCE, Anekal

 REGULATION 2021 SCHEME DAA-21CS42

Divide and conquer methodology can be easily applied on the following problem.
1. Finding maximum and minimum
2. Merge sort
3. Quick sort
4. Binary Search

1. Finding maximum and minimum:

The problem is to find the maximum and minimum value in a set of ‘n’ elements.It can be solved by
divide and conquer technique.

n-1
F(n)=Σ2=2(n-1-1+1)=2(n-1)
1
Straight MaxMin requires 2(n-1) element comparisons in the best, average & worst cases. An
improvement in the algorithm can be done as follows:

We can replace the contents of the for loop by,

If (a [i]> Max) then Max = a [i] Else if (a [i]< Min ) then Min=a[i]. Number of comparisons is
reduced by half as n-1.

Divide and Conquer Solution:

a. Let P = (n, a [i],……,a [j]) denote an arbitrary instance of the problem.

Prepared By: Dr.Vinola.C, Prof.Saranya.C SSCE, Anekal

 REGULATION 2021 SCHEME DAA-21CS42

b. Here ‘n’ is the no. of elements in the list (a [i],….,a[j]) and we are interested in finding the
maximum and minimum of the list.

c. If the list has more than 2 elements, P has to be divided into smaller instances.

d. For example, we might divide ‘P’ into the 2 instances, P1=([n/2],a[1],……..a[n/2]) & P2= ( [n/2],
a[[n/2]+1],….., a[n]) After having divided ‘P’ into 2 smaller sub problems, we can solve them by
recursively invoking the same divide-and-conquer algorithm.

Algorithm:

Analysis:

Prepared By: Dr.Vinola.C, Prof.Saranya.C SSCE, Anekal

 REGULATION 2021 SCHEME DAA-21CS42

Complexity:

If T(n) represents this no., then the resulting recurrence relations is

T (n)=T([n/2]+T[n/2]+2 n>2

1 , n=2

1 , n=1

Solving recurrence relation:

T (n) = 2T(n/2) +2

= 2(2T(n/4)+2)+2

= 4T(n/4)+4+2

=22T(n/22)+22+2……………=2iT(n/2i)+2i+……+2

If n=2i+1,then T(n)=2iT(2)+2i+…..+2

T(n)=2i+2i+……+2

T(n)=n/2+2(2i-1)/(2-1)

=n/2+2i+1-2

=n/2+n-2

=3/2n-2

T(n) = (3n/2) – 2

Note that (3n/2) - 2 is the best-average and worst-case no. of comparisons.

2. Merge sort

Merge sort is based on divide-and-conquer technique. It sorts a given array A[0..n−1] by

dividing it into two halves sorting each of them recursively, and then merging the two smaller
sorted arrays into a single sorted one.

ALGORITHM Mergesort(A,low,high])
//Sorts array A[0..n − 1] by recursive mergesort

Prepared By: Dr.Vinola.C, Prof.Saranya.C SSCE, Anekal

 REGULATION 2021 SCHEME DAA-21CS42

//Input: An array A[0..n − 1] of orderable elements

//Output: Array A[0..n − 1] sorted in nondecreasing order
If(low<high)
mid=(low+high)/2
Merge(A, low,mid,high)
Mergesort(A,low,mid-1)
Mergesort(A,mid+1,high)
endif

ALGORITHM merge(A,low,mid,high)
//Merges two unsorted arrays to a sorted array
//Input: An array A[0..n − 1] of orderable elements

i=low
j=mid+1
k=low
while(i<=mid and j<=high)
if(a(i)<a(j))
b(k)=a(i)
i=i+1
k=k+1
else
b(k)=a(j)
j=j+1
k=k+1

endif
endwhile
while(i<=mid)
b(k)<-a(i)
k=k+1
i=i+1
Prepared By: Dr.Vinola.C, Prof.Saranya.C SSCE, Anekal
 REGULATION 2021 SCHEME DAA-21CS42

endwhile
while(j<=high)
b(k)<-a(j)
k=k+1
j=j+1
endwhile

For i=low to high

a(i)=b(i)

Recurrence relation:
(Refer to class notes for detailed solution)
The recurrence relation for the number of key comparisons C(n) is
C(n) = 2C(n/2) + Cmerge(n) for n > 1, C(1) = 0.

In the worst case, Cmerge(n) = n

Cworst(n) = 2Cworst(n/2) + n − 1 for n > 1, Cworst(1) = 0.

By Master Theorem, Cworst(n) ∈ Θ(n log n)

Prepared By: Dr.Vinola.C, Prof.Saranya.C SSCE, Anekal

 REGULATION 2021 SCHEME DAA-21CS42

3. Quick sort:
Quicksort is the other important sorting algorithm that is based on the divide-and-conquer
approach. quicksort divides input elements according to their value. A partition is an arrangement
of the array’s elements so that all the elements to the left of some element A[s] are less than or
equal to A[s], and all the elements to the right of A[s] are greater than or equal to it:

A[0] . . . A[s − 1] A[s] A[s + 1] . . . A[n − 1]

all are ≤A[s] all are ≥A[s]

Sort the two subarrays to the left and to the right of A[s] independently. No work required to
combine the solutions to the subproblems.
Here is pseudocode of quicksort:

//Sorts a subarray by quicksort

//Input: Subarray of array A[0..n − 1], defined by its left and right indices l and r
//Output: Subarray A[l..r] sorted in order
if l < r
s ←Partition(A[l..r]) //s is a split position

Prepared By: Dr.Vinola.C, Prof.Saranya.C SSCE, Anekal

 REGULATION 2021 SCHEME DAA-21CS42

Quicksort(A[l..s − 1])
Quicksort(A[s + 1..r])

ALGORITHM Partition(A[l..r])
//Partitions a subarray by using the first element as a pivot
//Input: Subarray of array A[0..n − 1], defined by its left and right indices l and r (l<r)
//Output: Partition of A[l..r], with the split position returned as this function’s value
p←A[l]
i ←l+1; j ←r
while(i<=j)
while(A[i]<=p)
i=i+1
while(A[j]>=P)
j=j-1
Swap(A(i),A(j))
endwhile
Swap(A[low], A[j ])
return j

Prepared By: Dr.Vinola.C, Prof.Saranya.C SSCE, Anekal

 REGULATION 2021 SCHEME DAA-21CS42

The number of key comparisons in the best case satisfies the recurrence
Cbest(n) = 2Cbest(n/2) + n for n > 1, Cbest(1) = 0.

By Master Theorem, Cbest(n) ∈ Θ(n log2 n); solving it exactly for n = 2k yields Cbest(n) = n log2 n.
The total number of key comparisons made will be equal to
Cworst(n) = (n + 1) + n + . . . + 3 = ((n + 1)(n + 2))/2− 3 ∈Θ(n2).

Prepared By: Dr.Vinola.C, Prof.Saranya.C

Page 11
 REGULATION 2021 SCHEME DAA-21CS42

4. Binary Search:
A binary search is efficient algorithm to find the position of a target (key) value within a sorted
array.

 The binary search algorithm begins by comparing the target value to the value of the middle
element of the sorted array. If the target value is equal to the middle element's value, then
the position is returned and the search is finished.
 If the target value is less than the middle element's value, then the search continues on the
lower half of the array.
 if the target value is greater than the middle element's value, then the search continues on
the upper half of the array.
 This process continues, eliminating half of the elements, and comparing the target value to
the value of the middle element of the remaining elements - until the target value is either
found (position is returned).

Binary search is a remarkably efficient algorithm for searching in a sorted array (Say A). It
works by comparing a search key K with the array’s middle element A[m]. If they match, the
algorithm stops; otherwise, the same operation is repeated recursively for the first half of the array if
K <A[m], and for the second half if K >A[m]:

Though binary search is clearly based on a recursive idea, it can be easily implemented as a
nonrecursive algorithm, too. Here is pseudocode of this nonrecursive version.

ALGORITHM BinarySearch(A[0..n − 1], K)

//Implements nonrecursive binary search
//Input: An array A[0..n − 1] sorted in ascending order and a search key K
//Output: An index of the array’s element that is equal to K/ or −1 if there is no such
element
l ← 0; r ← n − 1
while l ≤ r do

Prepared By: Dr.Vinola.C, Prof.Saranya.C

Page 12
 REGULATION 2021 SCHEME DAA-21CS42

m ← ‫ (ه‬l + r)/2]
if K = A[m] return m

else ifK <A[m]

r ←m−1
else l ← m + 1
return −1

The standard way to analyze the efficiency of binary search is to count the number of times the
search key is compared with an element of the array (three-way comparisons). One comparison of K
with A[m], the algorithm can determine whether K is smaller, equal to, or larger than A[m].
As an example, let us apply binary search to searching for K = 70 in the array. The iterations of
the algorithm are given in the following table:

3 14 27 31 39 42 55 70 74 81 85 93 98

For detailed Solution refer to class notes

Prepared By: Dr.Vinola.C, Prof.Saranya.C

Page 13
 REGULATION 2021 SCHEME DAA-21CS42

 Third, the logarithmic function grows so slowly that its values remain small even for very
large values of n.
The average case slightly smaller than that in the worst case
Cavg(n) ≈ log2 n
The average number of comparisons in a successful is

Cavg(n) ≈ log2 n − 1
The average number of comparisons in an unsuccessful is

Cavg(n) ≈ log2(n + 1).

Recursive Binary Search:

ALGORITHM binary_search ( array , start , end , x )

WHILE start < end

mid := ( start + end ) / 2

IF x = array(mid) THEN
RETURN mid
ELSE IF x > array[mid]

THEN return binary_search( array , mid + 1 , end , x )

ELSE return binary_search( array , start , mid - 1 , x )
END WHILE
return -1

Decrease and Conquer Approach:

Introduction:
The decrease-and-conquer technique is based on exploiting the relationship between a solution to a
given instance of a problem and a solution to its smaller instance. Once such a relationship is established,
it can be exploited either

Prepared By: Dr.Vinola.C, Prof.Saranya.C

Page 14
 REGULATION 2021 SCHEME DAA-21CS42

top down or bottom up. The former leads naturally to a recursive implementation and The bottom-up
variation is usually implemented iteratively, starting with a solution to the smallest instance of the
problem; it is called sometimes the incremental approach.
There are three major variations of decrease-and-conquer:
 decrease by a constant
 decrease by a constant factor
 variable size decrease

Decrease by constant:
In the decrease-by-a-constant variation, the size of an instance is reduced by the same constant on each
iteration of the algorithm.
Consider, as an example, the exponentiation problem of computing an where a = 0 and n is a non negative
integer. The relationship between a solution to an instance of size n and an instance of size n − 1 is
obtained by the obvious formula an = an−1 . a. So the function f (n) = an can be computed either “top down”
by using its recursive definition.

Prepared By: Dr.Vinola.C, Prof.Saranya.C

Page 15
 REGULATION 2021 SCHEME DAA-21CS42

The decrease-by-a-constant-factor technique suggests reducing a problem instance by the

same constant factor on each iteration of the algorithm. In most applications, this constant
factor is equal to two.

Finally, in the variable-size-decrease variety of decrease-and-conquer, the size-reduction pattern varies

from one iteration of an algorithm to another. Euclid’s algorithm for computing the greatest common
divisor provides a good example of such a situation.

INSERTION sort:

we consider an application of the decrease-by-one technique to sorting an array A[0..n − 1]. Following
the technique’s idea, we assume that the smaller problem of sorting the array A[0..n −2] has already been
solved to give us a sorted array of size n −1: A[0]≤ . . . ≤ A[n −2]. How can we take advantage of this
solution to the smaller problem to get a solution to the original problem by taking into account the
element A[n −1]? Obviously, all we need is to find an appropriate position for A[n −1] among the sorted
elements and insert it there.

Prepared By: Dr.Vinola.C, Prof.Saranya.C

Page 16
 REGULATION 2021 SCHEME DAA-21CS42

This is usually done by scanning the sorted subarray from right to left until the first element smaller than
or equal to A[n −1] is encountered to insert A[n −1] right after that element. The resulting algorithm is
called straight insertion sort or simply insertion sort.

altogether.)
The number of key comparisons in this algorithm obviously depends on the nature of the input. In the
worst case, A[j ]> v is executed the largest number of times, i.e., for every j = i − 1, . . . , 0. Since v = A[i],
it happens if and only if A[j ]>A[i] for j = i − 1, . . . , 0. (Note that we are using the fact that on the ith
iteration of insertion sort all the elements preceding A[i] are the first i elements in the input, albeit in the
sorted order.) Thus, for the worst-case input, we get A[0]> A[1] (for i = 1), A[1]>A[2] (for i = 2), . . . ,
A[n − 2]>A[n − 1] (for i = n − 1).
In other words, the worst-case input is an array of strictly decreasing values. The number of key
comparisons for such an input is

Prepared By: Dr.Vinola.C, Prof.Saranya.C

Page 17
 REGULATION 2021 SCHEME DAA-21CS42

In the best case, the comparison A[j ] > v is executed only once on every iteration of the outer loop. It
happens if and only if A[i − 1] ≤ A[i] for every i = 1,...,n − 1, i.e., if the input array is already sorted in
nondecreasing order.

Graph Searching algorithms:

Depth First Search(DFS):
Depth-first search is an algorithm for traversing or searching tree or graph data structures. The algorithm
starts at the root node (selecting some arbitrary node as the root node in the case of a graph) and explores
as far as possible along each branch before backtracking.
Let us understand the working of Depth First Search with the help of the following illustration:
Step1: Initially stack and visited arrays are empty.

Step 2: Visit 0 and put its adjacent nodes which are not visited yet into the stack.

Step 3: Now, Node 1 at the top of the stack, so visit node 1 and pop it from the stack and put all of its
adjacent nodes which are not visited in the stack.

Prepared By: Dr.Vinola.C, Prof.Saranya.C

Page 18
 REGULATION 2021 SCHEME DAA-21CS42

Step 4: Now, Node 2 at the top of the stack, so visit node 2 and pop it from the stack and put all of its
adjacent nodes which are not visited (i.e, 3, 4) in the stack.

Step 5: Now, Node 4 at the top of the stack, so visit node 4 and pop it from the stack and put all of its
adjacent nodes which are not visited in the stack.

Step 6: Now, Node 3 at the top of the stack, so visit node 3 and pop it from the stack and put all of its
adjacent nodes which are not visited in the stack.

Prepared By: Dr.Vinola.C, Prof.Saranya.C

Page 19
 REGULATION 2021 SCHEME DAA-21CS42

Breadth First Search:

Starting from the root, all the nodes at a particular level are visited first and then the nodes of the next
level are traversed till all the nodes are visited.
To do this a queue is used. All the adjacent unvisited nodes of the current level are pushed into the
queue and the nodes of the current level are marked visited and popped from the queue.

Step1: Initially queue and visited arrays are empty.

Queue and visited arrays are empty initially.

Step2: Push node 0 into queue and mark it visited.

Prepared By: Dr.Vinola.C, Prof.Saranya.C

Page 20
 REGULATION 2021 SCHEME DAA-21CS42

Push node 0 into queue and mark it visited.

Step 3: Remove node 0 from the front of queue and visit the unvisited neighbours and push them into
queue.

Remove node 0 from the front of queue and visited the unvisited neighbours and push into queue.
Step 4: Remove node 1 from the front of queue and visit the unvisited neighbours and push them into
queue.

Remove node 1 from the front of queue and visited the unvisited neighbours and push
Step 5: Remove node 2 from the front of queue and visit the unvisited neighbours and push them into
queue.

Step 6: Remove node 3 from the front of queue and visit the unvisited neighbours and push them into
queue.

Prepared By: Dr.Vinola.C, Prof.Saranya.C

Page 21
 REGULATION 2021 SCHEME DAA-21CS42

As we can see that every neighbours of node 3 is visited, so move to the next node that are in the front
of the queue.

Remove node 3 from the front of queue and visit the unvisited neighbours and push them into queue.
Steps 7: Remove node 4 from the front of queue and visit the unvisited neighbours and push them into
queue.
As we can see that every neighbours of node 4 are visited, so move to the next node that is in the front
of the queue.

Remove node 4 from the front of queue and visit the unvisited neighbours and push them into queue.
Now, Queue becomes empty, So, terminate these process of iteration.

Topological sort:

Prepared By: Dr.Vinola.C, Prof.Saranya.C

Page 22
 REGULATION 2021 SCHEME DAA-21CS42

Background: A directed graph, or digraph for short, is a graph with directions specified for all its edges.
The adjacency matrix and adjacency lists are the two principal means of representing a digraph Ther
are only two notable differences between undirected and directed graphs in representing them:
(1) the adjacency matrix of a directed graph does not have to be symmetric; (2) an edge in a directed
graph has just one (not two) corresponding nodes in the digraph’s adjacency lists.
Depth-first search and breadth-first search are principal traversal algorithms for traversing digraphs as
well, but the structure of corresponding forests can be more complex than for undirected graphs. Thus,
even for the simple example of Figure, the depth-first search forest (Figure b) exhibits all four types of
edges possible in a DFS forest of a directed graph:
tree edges (ab, bc, de),
back edges (ba) from vertices to their ancestors,
forward edges (ac) from vertices to their descendants in the tree other than their children, and

cross edges (dc), which are none of the aforementioned types

Note that a back edge in a DFS forest of a directed graph can connect a vertex to its parent. Whether or
not it is the case, the presence of a back edge indicates that the digraph has a directed cycle. A directed
cycle in a digraph is a sequence of three or more of its vertices that starts and ends with the same vertex
and in which every vertex is connected to its immediate predecessor by an edge directed from the
predecessor to the successor. For example, a, b, a is a directed cycle in the digraph in Figure given
above. Conversely, if a DFS forest of a digraph has no backedges, the digraph is a dag, an acronym for
directed acyclic graph.

Motivation for topological sorting: Consider a set of five required courses {C1, C2, C3, C4,C5} a part-
time student has to take in some degree program. The courses can betaken in any order as long as the
following course prerequisites are met: C1 andC2 have no prerequisites, C3 requires C1 and C2, C4
requires C3, and C5 requiresC3 and C4. The student can take only one course per term. In which order
should the student take the courses?The situation can be modeled by a digraph in which vertices
represent courses and directed edges indicate prerequisite requirements.

Prepared By: Dr.Vinola.C, Prof.Saranya.C

Page 23
 REGULATION 2021 SCHEME DAA-21CS42

In terms of this digraph, the question is whether we can list its vertices in such an order that for every
edge in the graph, the vertex where the edge starts is listed before the vertex where the edge ends. In
other words, can you find such an ordering of this digraph’s vertices? This problem is called
topological sorting.

Topological Sort: For topological sorting to be possible, a digraph in question must be a DAG. i.e., if a
digraph has nodirected cycles, the topological sorting problem for it has a solution.

There are two efficient algorithms that both verify whether a digraph is a dag and, if it is, produce an
ordering of vertices that solves the topological sortingproblem.The first one is based on depth-first
search; the second is based on a direct application of the decrease-by-one technique.

Topological Sorting based on DFS Method

Perform a DFS traversal and note the order in which vertices become dead-ends

Reversing this order yields a solution to the topological sorting problem, provided, of course, no back
edge has been encountered during the traversal. If a back edge has been encountered, the digraph is not
a DAG, and topological sorting of its vertices is impossible.

Illustration

Digraph for which the topological sorting problem needs to be solved.

DFS traversal stack w th the subscript numbers indicating the popping off order.

Solution to the problem. Here we have drawn the edges of the digraph, andthey all point from left to
right as the problem’s statement requires. It is a convenientway to check visually the correctness of a
solution to an instance of thetopological sorting problem.

Topological Sorting using Source Removal Method:

Method: The algorithm is based on a direct implementation of the decrease-(byone)-and- conquer

technique

Prepared By: Dr.Vinola.C, Prof.Saranya.C

Page 24
 REGULATION 2021 SCHEME DAA-21CS42

Repeatedly, identify in a remaining digraph a source, which is a vertex with no incoming edges
and delete it along with all the edges outgoing from it. (If there are several sources, break the tie
arbitrarily. If there are none, stop because the problem cannot be solved.)

The order in which the vertices are deleted yields a solution to the topological sorting problem.

Illustration - Illustration of the source-removal algorithm for the topological sortingproblem is given
here. On each iteration, a vertex with no incoming edges is deletedfrom the digraph

Note: The solution obtained by the source-removal algorithm is differentfrom the one obtained by the
DFS-based algorithm. Both of them are correct, ofcourse; the topological sorting problem may have
several alternative solutions

Applications of Topological Sorting

 Instruction scheduling in program compilation

 Cell evaluation ordering in spreadsheet formulas,

 Resolving symbol dependencies in linkers

Note:Check the algorithms for topological sort in class notes

Prepared By: Dr.Vinola.C, Prof.Saranya.C

Page 25