Unit 2 - DAA - AK2
Unit 2 - DAA - AK2
Divide-and-conquer
Dr. S. Nagendra Prabhu
Syllabus
• Introduction, divide-and-conquer,
• Binary Search and its algorithm analysis
• Merge sort and its algorithm analysis
• Quick sort and its algorithm analysis
• Strassen's Matrix multiplication
• Finding Maximum and minimum
• Algorithm for finding closest pair
• Convex Hull Problem
Introduction
• Divide and Conquer is an algorithmic pattern. In
algorithmic methods, the design is to take a dispute
on a huge input, break the input into minor pieces,
decide the problem on each of the small pieces, and
then merge the piecewise solutions into a global
solution. This mechanism of solving the problem is
called the Divide & Conquer Strategy.
• Divide and Conquer algorithm consists of a dispute
using the following three steps.
1.Divide the original problem into a set of
subproblems.
2.Conquer: Solve every subproblem individually,
recursively.
3.Combine: Put together the solutions of the
subproblems to get the solution to the whole
problem.
Introduction
• Divide / Break
• Conquer / Solve
• Merge / Combine
Examples : Divide and Conquer Technique
General Method:
The Divide and Conquer Technique splits n inputs
into k subsets , 1< k ≤ n, yielding k subproblems.
Algorithm DAndC(p)
{
if Small(p) then return s(p);
else
{
divide p into smaller instances p1,p2,…….,pk, k≥1;
Apply DAndC to each of these subproblems;
return Combine(DAndC(p1), DAndC(p2),……,DAndC(pk));
}
}
General format
If the size of p is n and the sizes of the k subproblems are n1,n2,
….,nk,then the computing time of DAndC is described by the
recurrence relation
T(n)= g( n) n small
T(n1)+T(n2)+……+T(nk)+f(n) Otherwise
Where T(n) is the time for DAndC on any input of size n and g(n) is
the time to compute the answer directly for small inputs.
The function f(n) is the time for dividing p and combining the
solutions to subproblems.
Binary Search
• Divide / Break
• Conquer / Solve
• Merge / Combine
Binary Search
1. In Binary Search technique, we search an element
in a sorted array by recursively dividing the interval
in half.
2. Firstly, we take the whole array as an interval.
3. If the Pivot Element (the item to be searched) is
less than the item in the middle of the interval, We
discard the second half of the list and recursively
repeat the process for the first half of the list by
calculating the new middle and last element.
4. If the Pivot Element (the item to be searched) is
greater than the item in the middle of the interval,
we discard the first half of the list and work
recursively on the second half by calculating the
new beginning and middle element.
5. Repeatedly, check until the value is found or
interval is empty.
BINARY SEARCH
• Binary search is implemented using following steps...
• Step 1 - Read the search element from the user.
• Step 2 - Find the middle element in the sorted list.
• Step 3 - Compare the search element with the middle element in the sorted list.
• Step 4 - If both are matched, then display "Given element is found!!!" and terminate the
function.
• Step 5 - If both are not matched, then check whether the search element is smaller or
larger than the middle element.
• Step 6 - If the search element is smaller than middle element, repeat steps 2, 3, 4 and 5
for the left sublist of the middle element.
• Step 7 - If the search element is larger than middle element, repeat steps 2, 3, 4 and 5 for
the right sublist of the middle element.
• Step 8 - Repeat the same process until we find the search element in the list or until
sublist contains only one element.
• Step 9 - If that element also doesn't match with the search element, then display
"Element is not found in the list!!!" and terminate the function.
How Binary Search Works?
• Here it is, 0 + (9 - 0 ) / 2 = 4 (integer value of 4.5). So, 4 is the mid of the array.
How Binary Search Works?
• We change our low to mid + 1 and find the new mid value again.
low = mid + 1
mid = Big + end / 2
Divide into
A FirstPart SecondPart
two halves
Recursively
Recursively sort
sort
FirstPart SecondPart
Merge
A is sorted!
Merge Sort: Algorithm
MergeSort ( low,high)
// sorts the elements a[low],…,a[high] which reside in the global array
//a[1:n] into ascending order.
// Small(p) is true if there is only one element to sort. In this case the list is
// already sorted.
{
Recursive Calls
mid ← (low+high)/2;
MergeSort(low,mid);
MergeSort(mid+1, high);
Merge(low, mid, high);
}
Merge(A,L,M,H)
Merge Algorithm
{
Split the array into two array a and b;
i=0;j=0;k=0;
for(k=1;k<size[A];k++)
{
If (a[i]<b[j])
{
list[k]=a[i];
i++;
}
else
{
list[k]=b[j];
j++;
}
} list[k] = remaining_elements;
}
Merge-Sort: Merge Example
low mid high
A: 2
5 3
5 7 28
15 8 30
1 4
6 5 14
10 6
B: 5 5 15 28 30 6 10 14
L: R:
3 5 15 28 6 10 14 22
Merge-Sort: Merge Example
A: 5 5 15 28 30 6 10 14
B:
3
1 5 15 28 30 6 10 14
k=low
L: R:
3
2 15
3 28
7 30
8 6
1 10
4 14
5 22
6
i=low j=mid+1
Merge-Sort: Merge Example
A: 5 5 15 28 30 6 10 14
B:
1 2
5 15 28 30 6 10 14
L: R:
3
2 5
3 15
7 28
8 6
1 10
4 14
5 22
6
i j
Merge-Sort: Merge Example
A: 5 5 15 28 30 6 10 14
B:
1 2 3 28 30
15 6 10 14
L: R:
2 3 7 8 6
1 10
4 14
5 22
6
i j
Merge-Sort: Merge Example
A: 5 5 15 28 30 6 10 14
B:
1 2 3 4 6 10 14
L: R:
2 3 7 8 6
1 10
4 14
5 22
6
i j
Merge-Sort: Merge Example
A: 5 5 15 28 30 6 10 14
B:
1 2 3 4 5 6 10 14
L: R:
2 3 7 8 6
1 10
4 14
5 22
6
i j
Merge-Sort: Merge Example
A: 5 5 15 28 30 6 10 14
B:
1 2 3 4 5 6 10 14
L: R:
2 3 7 8 6
1 10
4 14
5 22
6
i j
Merge-Sort: Merge Example
A: 5 5 15 28 30 6 10 14
B:
1 2 3 4 5 6 7 14
L: R:
2 3 7 8 6
1 10
4 14
5 22
6
i j
Merge-Sort: Merge Example
A: 5 5 15 28 30 6 10 14
B:
1 2 3 4 5 6 7 8
14
L: R:
3
2 5
3 15
7 28
8 6
1 10
4 14
5 22
6
i j
Merge-Sort: Merge Example
A: 5 5 15 28 30 6 10 14
B:
1 2 3 4 5 6 7 8
L: R:
3
2 5
3 15
7 28
8 6
1 10
4 14
5 22
6
i j
A:
5 5 15 28 30 6 10 14
B: 1 2 3 4 5 6 7 8
Merge-Sort Analysis
n
n/2 n/2
2 2 2
Analysis of Merge Sort:
• Note stopping condition T(1)=0 because at last, there will be only 1 element
that need to be copied, and there will be no comparison.
• So, we get,
Merge-Sort Time Complexity
If the time for the merging operation is proportional to n, then the
computing time for merge sort is described by the recurrence relation
c1 n=1, c1 is a constant
T(n) =
2T(n/2) + c2n n>1, c2 is a constant
…..
…..
=2k T(1)+ kc2n
= c1n+c2nlogn = = O(nlogn)
Summary
• Merge-Sort
– Most of the work done in combining the
solutions.
– Best case takes o(n log(n)) time
– Average case takes o(n log(n)) time
– Worst case takes o(n log(n)) time
Quick Sort
3. Quick Sort
• Divide:
• Pick any one of the following as pivot,
– First element
– Last element
– Middle element
– Random element
• Partition the remaining elements into
FirstPart, which contains all elements < pivot
SecondPart, which contains all elements > pivot
4 2 7 8 1 9 3 6 5
pivot
x y
The whole process
4 2 7 8 1 9 3 6 5
2 1 3 4 7 8 9 6 5
1 2 3 6 5 7 8 9
1 3 5 6 8 9
5 9
Quick Sort Algorithm :
Algorithm QuickSort(low,high)
//Sorts the elements a[low],…..,a[high] which resides
//in the global array a[1:n] into ascending order;
// a[n+1] is considered to be defined and must ≥ all the
// elements in a[1:n].
{
if( low< high ) // if there are more than one element
{ // divide p into two subproblems.
j :=Partition(low,high);
// j is the position of the partitioning element.
QuickSort(low,j-1);
QuickSort(j+1,high);
// There is no need for combining solutions.
}
}
Algorithm Partition(l,h)
{
pivot:= a[l] ; i:=l; j:= h+1;
while( i < j ) do
{
i++;
while( a[ i ] < pivot ) do
i++;
j--;
while( a[ j ] > pivot ) do
j--;
n/2 n/2
2 2 2
• Prove O(nlogn)
QUICK SORT
•
Time complexity analysis
A worst/bad case
1 2 3 4 5 6 7 8 9
1 2 3 4 5 6 7 8 9
2 3 4 5 6 7 8 9
3 4 5 6 7 8 9
4 5 6 7 8 9
5 6 7 8 9 O(n2)
6 7 8 9
7 8 9
8 9
9
Worst/bad Case
n cn
n-1 c(n-1)
n-2 c(n-2)
3c
3
Happens only if
• input is sorted 2 2c
• input is reversely sorted 1c
1
Total time: O(n2)
T (n) =T(n-1)+n
c11 c12 1 1 2 2 5 5 6 6
C11=A11B11+A12B21 C11 C12 1 1 2 2 5 5 6 6
C
C12=A11B12+A12B22 C 21 C 22 3 3 4 4 7 7 8 8
c21 c22 3 3 4 4 7 7 8 8
C21=A21B11+A22B21 A 21 A 22 B 21 B22
C22=A21B12+A22B22
• Each of these four equations specifies two multiplications of n/2×n/2
matrices and the addition of their n/2×n/2 products.
• We can derive the following recurrence relation for the time T(n) to multiply
two n×n matrices:
T(n)= c1 if n<=2
8T(n/2)+ c2n2 if n>2
T(n) = O(n3)
Strassen’s method
M1 + M4 - M5 + M7 M 3 + M5
=
M2 + M4 M1 + M3 - M2 + M6
T(n)= c1 n<=2
7T(n/2) +c2n2 n>2
T(n)= 7kT(1) + c2n2 1+ 7/4 + (7/4)2 + (7/4) 3+……………..+ (7/4)k-1
..
.
=
7log2n c1 +c2 n2 (7/4)log2n
=
nlog27 + nlog24 ( n log27-log24 )
Complexity
best= average =worst= 2(n-1) comparisons
56 34 12 1 76 34 23 8 16
56 34 12 1 76 34 23 8 16
56 34 12 1 76 34 23 8 16
Min 8
Max 76
Min 1
Max 76
Algorithm
• Algorithm: max_min(x, y)
if y – x ≤ 1 then
return (max(numbers[x], numbers[y]), min((numbers[x], numbers[y]))
else
Mid=(l+h)/2
(max1, min1):= max_min(x, mid)
(max2, min2):= max_min(mid+1,y)
return (max(max1, max2), min(min1, min2))
Complexity Analysis
1.if size= 1 return current element as both max and min //base condition
2.else if size= 2 one comparison to determine max and min //base condition
recur for max and min of left half recur for max and min of right half.
By introducing
• Best Cases
Algorithm for finding closest pair
• Given a set of points, find the closest pair
(measured in Euclidean distance).
• a set of n points are given on the 2D
plane. In this problem, we have to find the
pair of points, whose distance is minimum.
• The Brute force solution is O(n2), compute
the distance between each pair and return
the smallest.
• We can calculate the smallest distance
in ? time using Divide and Conquer
strategy.
Algorithm
Input: An array of n points P[]
Output: The smallest distance between two points in the given array.
As a pre-processing step, input array is sorted with x coordinates.
1) Find the middle point in the sorted array, we can take P[n/2] as middle
point.
2) Divide the given array in two halves. The first subarray contains points
from P[0] to P[n/2]. The second subarray contains points from P[n/2+1] to
P[n-1].
3) Recursively find the smallest distances in both subarrays. Let the
distances be dl and dr. Find the minimum of dl and dr. Let the minimum be
d.
4) From above 3 steps, we have an upper bound d of minimum distance.
Now we need to consider the pairs such that one point in pair is from
left half and other is from right half. Consider the vertical line passing
through passing through P[n/2] and find all points whose x coordinate is
closer than d to the middle vertical line. Build an array strip[] of all such
points.
5) Sort the array strip[] according to y coordinates. This step is O(nLogn). It
can be optimized to O(n) by recursively sorting and merging.
6) Find the smallest distance in strip[]. This is tricky. From first look, it seems
to be a O(n2) step, but it is actually O(n).
3. If f(n) > Ω (nlogba), and f(n) satisfies the condition, then T (n) = ϴ (f(n)).
Compare with f(n). Since f(n) < nlogba i.e. nlogn = n1log1n