DAA Module - 2
DAA Module - 2
Contents
1. General method
2. Recurrence equation
3. Algorithm: Finding the maximum and minimum
4. Algorithm: Binary search
5. Algorithm: Merge sort
6. Algorithm: Quick sort
7. Decrease and Conquer Approach
8. Algorithm: Insertion sort
9. Graph searching algorithm
10. Algorithm: Topological Sort
1. General method
Divide and Conquer is one of the best-known general algorithm design technique. It works
according to the following general plan:
• Given a function to compute on 'n' inputs the divide-and-conquer strategy suggests
splitting the inputs into 'k' distinct subsets, 1<k<=n, yielding 'k' sub problems.
• These sub problems must be solved, and then a method must be found to combine sub
solutions into a solution of the whole.
• If the sub problems are still relatively large, then the divide-and-conquer strategy can
possibly be reapplied.
• Often the sub problems resulting from a divide-and-conquer design are of the same
type as the original problem. For those cases the reapplication of the divide-and-
conquer principle is naturally expressed by a recursive algorithm.
Problem of n size
T(n)=¿
where f(n) is a function that accounts for the time spent on dividing the problem into smaller
ones and on combining their solutions.
The recurrence relation can be solved by i) substitution method or by using ii) master
theorem.
1. Substitution Method - This method repeatedly makes substitution for each occurrence
of the function T in the right-hand side until all such occurrences disappear.
2. Master Theorem - The efficiency analysis of many divide-and-conquer algorithms is
greatly simplified by the master theorem. It states that, in recurrence equation T(n) =
aT(n/b) + f (n), If f (n)E 0 (nd) where d 2: 0 then
{
θ ( n ) if a< b
d d
Problems on Substitution method & Master theorem to solve the recurrence relation
4
5
6
7
8
3. Finding the maximum and minimum
Problem statement: Given a list of n elements, the problem is to find the maximum and
minimum items.
StraightMaxMin: A simple and straight forward algorithm to achieve this is given below.
Explanation:
■ StraightMaxMin requires 2(n-1) comparisons in the best, average & worst cases.
■ By realizing the comparison of a[i]>max is false, improvement in a algorithm can be
done. Hence we can replace the contents of the for loop by,
If(a[i]>Max) then Max= a[i]; Else if (a[i]<min) min=a[i]
■ On the average a[i] is > max half the time. So, the avg. no. of comparison is 3n/2-1.
11
Recursive binary search algorithm
12
{
int low= 1, high= n;
while (low<= high){
int mid= (low+ high)/2;
if (x-< a[mid]) high= mid - 1;
else if (x > a[mid]) low= mid+ 1;
else return(mid);
}
return(O);
}
-15, -6, 0, 7, 9, 23, 54, 82, 101, 112, 125, 131, 142, 151
place them in a[l : 14], and simulate the steps that BinSearch goes through as it searches
for different values of x. Only the variables low, high, and mid need to be traced as we
simulate the algorithm. We try the following values for x: 151, -14, and9 for two successful
searches and one unsuccessful search.
Table shows the traces of BinSearch on these three inputs.
Analysis
In binary search the basic operation is key comparison. Binary Search can be analyzed with
the best, worst, and average case number of comparisons. The numbers of comparisons for
the recursive and iterative versions of Binary Search are the same, if comparison counting is
relaxed slightly. For Recursive Binary Search, count each pass through the if-then-else block
as one comparison. For Iterative Binary Search, count each pass through the while block as
one comparison. Let us find out how many such key comparison does the algorithm make on
an array of n elements.
Best case - Ɵ(1) In the best case, the key is the middle in the array. A constant number of
comparisons (actually just 1) are required.
Worst case - Ɵ(log2 n) In the worst case, the key does not exist in the array at all. Through
each recursion or iteration of Binary Search, the size of the admissible range is halved. This
13
halving can be done ceiling (log2 n) times. Thus, [ log2 n] comparisons are required.
14
Sometimes, in case of the successful search, it may take maximum number of comparisons.
[log2 n] . So worst case complexity of successful binary search is Ɵ(log2 n).
Average case - Ɵ(log2n) To find the average case, take the sum of the product of number of
comparisons required to find each element and the probability of searching for that element.
To simplify the analysis, assume that no item which is not in array will be searched for, and
that the probabilities of searching for each element are uniform.
successful searches unsuccessful searches
Ɵ(1), Ɵ(logn), Ɵ(logn) Ɵ(logn)
best, average, worst best, average, worst
Space Complexity - The space requirements for the recursive and iterative versions of
binary search are different. Iterative Binary Search requires only a constant amount of space,
while Recursive Binary Search requires space proportional to the number of comparisons to
maintain the recursion stack.
Advantages: Efficient on very big list, can be implemented iteratively/recursively.
Limitations:
• Interacts poorly with the memory hierarchy
• Requires sorted list as an input
• Due to random access of list element, needs arrays instead of linked list.
5. Merge Sort
Merge sort is a perfect example of a successful application of the divide-and conquer technique.
It sorts a given array A [O ... n - 1] by dividing it into two halves A [O .. n/2-1] and A [n/2
.. n-1], sorting each of them recursively, and then merging the two smaller sorted arrays into a
single sorted one.
15
two given arrays is exhausted, and then the remaining elements of the other array are
copied to the end of the new array.
16
17
Advantages:
• Number of comparisons performed is nearly optimal.
• For large n, the number of comparisons made by this algorithm in the average case
turns out to be about 0.25n less and hence is also in B(n log n).
• Merge sort will never degrade to O (n2)
• Another advantage of merge sort over quicksort and heapsort is its stability. (A
sorting algorithm is said to be stable if two objects with equal keys appear in the
same order in sorted output as they appear in the input array to be sorted. )
Limitations: The principal shortcoming of merge sort is the linear amount [O(n)] of extra
storage the algorithm requires. Though merging can be done in-place, the resulting algorithm
is quite complicated and of theoretical interest only.
Variations of merge sort:
1. The algorithm can be implemented bottom up by merging pairs of the array's elements,
then merging the sorted pairs, and so on. This avoids the time and space overhead of
using a stack to handle recursive calls.
2. We can divide a list to be sorted in more than two parts, sort each recursively, and then
merge them together. This scheme, which is particularly useful for sorting files residing
on secondary memory devices, is called multi way merge sort.
6. Quick sort
Quicksort is the other important sorting algorithm that is based on the divide-and-conquer
approach. Unlike merge sort, which divides its input elements according to their position in
the array, quicksort divides ( or partitions) them according to their value.
A partition is an arrangement of the array's elements so that all the elements to the left of
some element A[s] are less than or equal to A[s], and all the elements to the right of A[s] are
greater than or equal to it:
Obviously, after a partition is achieved, A[s] will be in its final position in the sorted array,
and we can continue sorting the two subarrays to the left and to the right of A[s]
independently (e.g., by the same method).
In quick sort, the entire work happens in the division stage, with no work required to
combine the solutions to the sub problems.
Partitioning
We start by selecting a pivot-an element with respect to whose value we are going to divide
the subarray. There are several different strategies for selecting a pivot.
Select the subarray's first element: p = A[l]. Now scan the subarray from both ends,
comparing the subarray' s elements to the pivot.
■ The left-to-right scan, denoted below by index pointer i, starts with the second
element. Since we want elements smaller than the pivot to be in the left part of the
subarray, this scan skips over elements that are smaller than the pivot and stops upon
encountering the first element greater than or equal to the pivot.
■ The right-to-left scan, denoted below by index pointer j, starts with the last element of
the subarray. Since we want elements larger than the pivot to be in the right part of
the subarray, this scan skips over elements that are larger than the pivot and stops on
encountering the first element smaller than or equal to the pivot.
After both scans stop, three situations may arise, depending on whether or not the scanning
indices have crossed.
Analysis
Best Case -Here the basic operation is key comparison. Number of key comparisons made
before a partition is achieved is n + 1 if the scanning indices cross over and n if they coincide.
If all the splits happen in the middle of corresponding subarrays, we will have the best case.
The number of key comparisons in the best case satisfies the recurrence,
Worst Case - In the worst case, all the splits will be skewed to the extreme: one of the two
subarrays will be empty, and the size of the other will be just I less than the size of the
subarray being partitioned. This unfortunate situation will happen, in particular, for
increasing arrays. Indeed, if A[0..n - 1] is a strictly increasing array and we use A[0] as the
pivot, the left-to-right scan will stop on A[l] while the right-to-left scan will go all the way to
reach A[0], indicating the split at position 0:So, after making n + 1 comparisons to get to this
partition and exchanging the pivot A[0] with itself, the algorithm will be left with the strictly
increasing array A[l..n - l] to sort. This sorting of strictly increasing arrays of diminishing
sizes will continue until the last one A[n-2.. n-1] has been processed. The total number of key
comparisons made will be equal to
Thus, on the average, quicksort makes only 39% more comparisons than in the best case.
Moreover, its innermost loop is so efficient that it usually runs faster than merge sort on
randomly ordered arrays of nontrivial sizes. This certainly justifies the name given to the
algorithm by its inventor.
Variations: Because of quicksort's importance, there have been persistent efforts over the
years to refine the basic algorithm. Among several improvements discovered by researchers
are:
■ Better pivot selection methods such as randomized quicksort that uses a random
element or the median-of-three method that uses the median of the leftmost,
rightmost, and the middle element of the array
■ Switching to insertion sort on very small subarrays (between 5 and 15 elements for
most computer systems) or not sorting small subarrays at all and finishing the algorithm
with insertion sort applied to the entire nearly sorted array
■ Modifications of the partitioning algorithm such as the three-way partition into
segments smaller than, equal to, and larger than the pivot
Limitations:
1. It is not stable.
2. It requires a stack to store parameters of subarrays that are yet to be sorted.
3. While Performance on randomly ordered arrays is known to be sensitive not only to
the implementation details of the algorithm but also to both computer architecture and
data type.
Time Complexities:
o Best Case Complexity: The insertion sort algorithm has a best-case time complexity of O(n) for
the already sorted array because here, only the outer loop is running n times, and the inner loop is
kept still.
o Average Case Complexity: The average-case time complexity for the insertion sort algorithm
is O(n2), which is incurred when the existing elements are in jumbled order, i.e., neither in the
ascending order nor in the descending order.
o Worst Case Complexity: The worst-case time complexity is also O(n2), which occurs when we
sort the ascending order of an array into the descending order.
In this algorithm, every individual element is compared with the rest of the elements, due to
which n-1 comparisons are made for every nth element.
The insertion sort algorithm is highly recommended, especially when a few elements are left for
sorting or in case the array encompasses few elements.
Space Complexity
The insertion sort encompasses a space complexity of O(1) due to the usage of an extra
variable key.
Terms:
There are many ways to traverse graphs. BFS is the most commonly used approach. BFS is a
traversing algorithm where you should start traversing from a selected node (source or starting node)
and traverse the graph layer wise thus exploring the neighbour nodes (nodes which are directly
connected to source node). You must then move towards the next-level neighbour nodes.
As the name BFS suggests, you are required to traverse the graph breadthwise as follows:
a) First move horizontally and visit all the nodes of the current layer.
b) Move to the next layer.
BFS traversal of a graph, produces a spanning tree as final result. Spanning Tree is a graph without
any loops. We use Queue data structure with maximum size of total number of vertices in the graph to
implement BFS traversal of a graph.
Step 2: Select any vertex as starting point for traversal. Visit that vertex and insert it into the
Queue.
Step 3: Visit all the adjacent vertices of the vertex which is at front of the Queue which is not
visited and insert them into the Queue.
Step 4: When there is no new vertex to be visit from the vertex at front of the Queue then
delete that vertex from the Queue.
Step 6: When queue becomes Empty, then produce final spanning tree by removing unused
edges from the graph
The DFS algorithm is a recursive algorithm that uses the idea of backtracking. It involves exhaustive
searches of all the nodes by going ahead, if possible, else by backtracking.
DFS traversal of a graph, produces a spanning tree as final result. Spanning Tree is a graph without
any loops. We use Stack data structure with maximum size of total number of vertices in the graph to
implement DFS traversal of a graph.
Step 2: Select any vertex as starting point for traversal. Visit that vertex and push it on to the
Stack.
Step 3: Visit any one of the adjacent vertex of the verex which is at top of the stack which is
not visited and push it on to the stack.
Step 4: Repeat step 3 until there are no new vertex to be visit from the vertex on top of the
stack.
Step 5: When there is no new vertex to be visit then use back tracking and pop one vertex
from the stack.
(a) (b)
(a) Digraph. (b) DFS forest of the digraph for the DFS traversal started at a.
Note that a back edge in a DFS forest of a directed graph can connect a vertex to its parent.
Whether or not it is the case, the presence of a back edge indicates that the digraph has a
directed cycle. A directed cycle in a digraph is a sequence of three or more of its vertices that
starts and ends with the same vertex and in which every vertex is connected to its immediate
predecessor by an edge directed from the predecessor to the successor. For example, a, b, a is
a directed cycle in the digraph in Figure given above. Conversely, if a DFS forest of a
digraph has no back edges, the digraph is a DAG, an acronym for directed acyclic graph.
Motivation for topological sorting: Consider a set of five required courses {Cl, C2, C3,
C4, C5} a part-time student has to take in some degree program. The courses can be taken
in any order as long as the following course prerequisites are met: C1 andC2 have no
prerequisites,C3 requires Cl and C2, C4 requires C3, and CS requiresC3 and C4. The student
can take only one course per term. In which order should the student take the courses? The
situation can be modeled by a digraph in which vertices represent courses and directed edges
indicate prerequisite requirements.
In terms of this digraph, the question is whether we can list its vertices in such an order that
for every edge in the graph, the vertex where the edge starts is listed before the vertex where
the edge ends. In other words, can you find such an ordering of this digraph's vertices? This
problem is called topological sorting.
Topological Sort: For topological sorting to be possible, a digraph in question must be a
DAG. i.e., if a digraph has no directed cycles, the topological sorting problem for it has a
solution.
There are two efficient algorithms that both verify whether a digraph is a DAG and, if it is,
produce an ordering of vertices that solves the topological sorting problem. The first one is
based on depth-first search; the second is based on a direct application of the decrease-by-one
technique.
Topological Sorting based on
DFS Method
1. Perform a DFS traversal and note the order in which vertices become dead-ends
2. Reversing this order yields a solution to the topological sorting problem, provided, of
course, no back edge has been encountered during the traversal. If a back edge has
been encountered, the digraph is not a DAG, and topological sorting of its vertices is
impossible.
Illustration
a) Digraph for which the topological sorting problem needs to be solved.
b) DFS traversal stack with the subscript numbers indicating the popping off order.
c) Solution to the problem. Here we have drawn the edges of the digraph, and they all
point from left to right as the problem's statement requires. It is a convenient way to
check visually the correctness of a solution to an instance of the topological sorting
problem.
Note: The solution obtained by the source-removal algorithm is different from the one
obtained by the DPS-based algorithm. Both of them are correct, of course; the topological
sorting problem may have several alternative solutions.
Applications of Topological Sorting