Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

CSE 221: Algorithms

Divide and Conquer

Mumit Khan

Computer Science and Engineering

BRAC University

References
1 T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to Algorithms, Second Edition.
The MIT Press, September 2001.
2 Erik Demaine and Charles Leiserson, 6.046J Introduction to Algorithms. MIT OpenCourseWare, Fall 2005.
Available from: ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Computer-Science/
6-046JFall-2005/CourseHome/index.htm

Last modified: June 11, 2009

This work is licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

Mumit Khan Licensed under CSE 221: Algorithms 1 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Divide-and-Conquer design strategy

1 Divide the problem (instance) into subproblems.

2 Conquer the subproblems by solving these recursively.
3 Combine the solutions to the subproblems.

Mumit Khan Licensed under CSE 221: Algorithms 2 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Divide-and-Conquer design strategy

1 Divide the problem (instance) into subproblems.

2 Conquer the subproblems by solving these recursively.
3 Combine the solutions to the subproblems.

Mumit Khan Licensed under CSE 221: Algorithms 2 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Divide-and-Conquer design strategy

1 Divide the problem (instance) into subproblems.

2 Conquer the subproblems by solving these recursively.
3 Combine the solutions to the subproblems.

Mumit Khan Licensed under CSE 221: Algorithms 2 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Divide-and-Conquer design strategy

1 Divide the problem (instance) into subproblems.

2 Conquer the subproblems by solving these recursively.
3 Combine the solutions to the subproblems.

Mumit Khan Licensed under CSE 221: Algorithms 2 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Divide-and-Conquer design strategy

1 Divide the problem (instance) into subproblems.

2 Conquer the subproblems by solving these recursively.
3 Combine the solutions to the subproblems.

Example (of D&C strategy)

1 Binary search – divide the problem into half, and recursively
search the appropriate 1 subproblem.
2 Mergesort – divide the problem into half, and recursively sort
2 subproblems, and then merge the results into a complete
sorted sequence.
3 Computing x n , computing fibonacci numbers, multiplying
matrices (using Strassen’s algorithm), etc.

Mumit Khan Licensed under CSE 221: Algorithms 2 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Binary search

The problem
Find an element in a sorted array:
1 Divide: Check the middle element.
2 Conquer: Recursively search 1 subarray.
3 Combine: Trivial.

Mumit Khan Licensed under CSE 221: Algorithms 3 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Binary search

The problem
Find an element in a sorted array:
1 Divide: Check the middle element.
2 Conquer: Recursively search 1 subarray.
3 Combine: Trivial.

Example (Searching for 9 )

Mumit Khan Licensed under CSE 221: Algorithms 3 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Binary search

The problem
Find an element in a sorted array:
1 Divide: Check the middle element.
2 Conquer: Recursively search 1 subarray.
3 Combine: Trivial.

Example (Searching for 9 )

Mumit Khan Licensed under CSE 221: Algorithms 3 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Binary search

The problem
Find an element in a sorted array:
1 Divide: Check the middle element.
2 Conquer: Recursively search 1 subarray.
3 Combine: Trivial.

Example (Searching for 9 )

Mumit Khan Licensed under CSE 221: Algorithms 3 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Binary search

The problem
Find an element in a sorted array:
1 Divide: Check the middle element.
2 Conquer: Recursively search 1 subarray.
3 Combine: Trivial.

Example (Searching for 9 )

Mumit Khan Licensed under CSE 221: Algorithms 3 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Binary search

The problem
Find an element in a sorted array:
1 Divide: Check the middle element.
2 Conquer: Recursively search 1 subarray.
3 Combine: Trivial.

Example (Searching for 9 )

Mumit Khan Licensed under CSE 221: Algorithms 3 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Binary search

The problem
Find an element in a sorted array:
1 Divide: Check the middle element.
2 Conquer: Recursively search 1 subarray.
3 Combine: Trivial.

Example (Searching for 9 )

Mumit Khan Licensed under CSE 221: Algorithms 3 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Recurrence for binary search

Mumit Khan Licensed under CSE 221: Algorithms 4 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Recurrence for binary search

Mumit Khan Licensed under CSE 221: Algorithms 4 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Recurrence for binary search

Analysis
nlogb a = nlog2 1 = n0 = 1 ⇒ CASE 2 (k = 0)
⇒ T (n) = Θ(nlogb a lgk+1 n) = Θ(lg n)

Mumit Khan Licensed under CSE 221: Algorithms 4 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merge sort

The problem
Sort an Array:
1 Divide: Trivial.
2 Conquer: Recursively sort 2 subarrays.
3 Combine: Merge the sorted subarrays in Θ(n) time.

Mumit Khan Licensed under CSE 221: Algorithms 5 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merge sort

The problem
Sort an Array:
1 Divide: Trivial.
2 Conquer: Recursively sort 2 subarrays.
3 Combine: Merge the sorted subarrays in Θ(n) time.

Key subroutine
merge – to merge two sorted arrays in linear-time.

Mumit Khan Licensed under CSE 221: Algorithms 5 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merge sort in action

Mumit Khan Licensed under CSE 221: Algorithms 6 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merge sort in action

Mumit Khan Licensed under CSE 221: Algorithms 6 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merge sort in action

Mumit Khan Licensed under CSE 221: Algorithms 6 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merge sort in action

Mumit Khan Licensed under CSE 221: Algorithms 6 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merge sort in action

Mumit Khan Licensed under CSE 221: Algorithms 6 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merge sort in action

Mumit Khan Licensed under CSE 221: Algorithms 6 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merge sort in action

Mumit Khan Licensed under CSE 221: Algorithms 6 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merging in Θ(n) time

Mumit Khan Licensed under CSE 221: Algorithms 7 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merging in Θ(n) time

Mumit Khan Licensed under CSE 221: Algorithms 7 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merging in Θ(n) time

Mumit Khan Licensed under CSE 221: Algorithms 7 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merging in Θ(n) time

Mumit Khan Licensed under CSE 221: Algorithms 7 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merging in Θ(n) time

Mumit Khan Licensed under CSE 221: Algorithms 7 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merging in Θ(n) time

Mumit Khan Licensed under CSE 221: Algorithms 7 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merging in Θ(n) time

Mumit Khan Licensed under CSE 221: Algorithms 7 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merging in Θ(n) time

Mumit Khan Licensed under CSE 221: Algorithms 7 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merging in Θ(n) time

Mumit Khan Licensed under CSE 221: Algorithms 7 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merging in Θ(n) time

Mumit Khan Licensed under CSE 221: Algorithms 7 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merging in Θ(n) time

Mumit Khan Licensed under CSE 221: Algorithms 7 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

A Θ(n) time merge algorithm

merge(A, B)
INPUT: Two sorted arrays A and B
OUTPUT: Returns C as the merged array
 n1 = length[A], n2 = length[B], n = n1 + n2
1 Create C [1 . . n]
2 Initialize two indices to point to A and B
3 while A and B are not empty
4 do Select the smaller of two and add to end of C
5 Advance the index that points to the smaller one
6 if A or B is not empty
7 then Copy the rest of the non-empty array to the end of C
8 return C

Mumit Khan Licensed under CSE 221: Algorithms 8 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

A Θ(n) time merge algorithm

merge(A, B)
INPUT: Two sorted arrays A and B
OUTPUT: Returns C as the merged array
 n1 = length[A], n2 = length[B], n = n1 + n2
1 Create C [1 . . n]
2 Initialize two indices to point to A and B
3 while A and B are not empty
4 do Select the smaller of two and add to end of C
5 Advance the index that points to the smaller one
6 if A or B is not empty
7 then Copy the rest of the non-empty array to the end of C
8 return C

T (n) = Θ(n)

Mumit Khan Licensed under CSE 221: Algorithms 8 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

A Θ(n) time merge algorithm

merge(A, B)
INPUT: Two sorted arrays A and B
OUTPUT: Returns C as the merged array
 n1 = length[A], n2 = length[B], n = n1 + n2
1 Create C [1 . . n]
2 Initialize two indices to point to A and B
3 while A and B are not empty
4 do Select the smaller of two and add to end of C
5 Advance the index that points to the smaller one
6 if A or B is not empty
7 then Copy the rest of the non-empty array to the end of C
8 return C

T (n) = Θ(n)

Issue: Out-of-place algorithm.

Mumit Khan Licensed under CSE 221: Algorithms 8 / 12
 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

A Θ(n) time merge algorithm

merge(A, B)
INPUT: Two sorted arrays A and B
OUTPUT: Returns C as the merged array
 n1 = length[A], n2 = length[B], n = n1 + n2
1 Create C [1 . . n]
2 Initialize two indices to point to A and B
3 while A and B are not empty
4 do Select the smaller of two and add to end of C
5 Advance the index that points to the smaller one
6 if A or B is not empty
7 then Copy the rest of the non-empty array to the end of C
8 return C

T (n) = Θ(n)

Issue: Out-of-place algorithm. Can it be made in-place?

Mumit Khan Licensed under CSE 221: Algorithms 8 / 12
 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merge sort algorithm

merge-sort(A)  A[1 . . n]
1 if n = 1
2 then return
3 else  recursively sort the two subarrays
4 A1 = merge-sort(A[1 . . dn/2e])
5 A2 = merge-sort(A[dn/2e] + 1 . . n])
6 A = merge(A1 , A2 )  merge the sorted arrays

Mumit Khan Licensed under CSE 221: Algorithms 9 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merge sort algorithm

merge-sort(A)  A[1 . . n]
1 if n = 1
2 then return
3 else  recursively sort the two subarrays
4 A1 = merge-sort(A[1 . . dn/2e])
5 A2 = merge-sort(A[dn/2e] + 1 . . n])
6 A = merge(A1 , A2 )  merge the sorted arrays

Few notes on the algorithm

1 Dividing is trivial, but it’s the merging that requires most time.
2 The merging algorithm presented here is an out-of-place
algorithm, which will increase space complexity.

Mumit Khan Licensed under CSE 221: Algorithms 9 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Merge sort algorithm

merge-sort(A)  A[1 . . n]
1 if n = 1
2 then return
3 else  recursively sort the two subarrays
4 A1 = merge-sort(A[1 . . dn/2e])
5 A2 = merge-sort(A[dn/2e] + 1 . . n])
6 A = merge(A1 , A2 )  merge the sorted arrays

Few notes on the algorithm

1 Dividing is trivial, but it’s the merging that requires most time.
2 The merging algorithm presented here is an out-of-place
algorithm, which will increase space complexity. Can it be
made in-place?

Mumit Khan Licensed under CSE 221: Algorithms 9 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Recurrence for merge sort

Mumit Khan Licensed under CSE 221: Algorithms 10 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Recurrence for merge sort

Mumit Khan Licensed under CSE 221: Algorithms 10 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Recurrence for merge sort

Analysis
nlogb a = nlog2 2 = n1 = n ⇒ CASE 2 (k = 0)
⇒ T (n) = Θ(nlogb a lgk+1 n) = Θ(n lg n)

Mumit Khan Licensed under CSE 221: Algorithms 10 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Powering a number

Problem: Compute an , where n ∈ N.

Naive algorithm: Θ(n).

Mumit Khan Licensed under CSE 221: Algorithms 11 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Powering a number

Problem: Compute an , where n ∈ N.

Naive algorithm: Θ(n).
Divide-and-conquer algorithm:
 n/2 n/2
a ·a if n is even
an = (n−1)/2 (n−1)/2
a ·a · a if n is odd

Mumit Khan Licensed under CSE 221: Algorithms 11 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Powering a number

Problem: Compute an , where n ∈ N.

Naive algorithm: Θ(n).
Divide-and-conquer algorithm:
 n/2 n/2
a ·a if n is even
an = (n−1)/2 (n−1)/2
a ·a · a if n is odd

T (n) = T (n/2) + Θ(1)

Mumit Khan Licensed under CSE 221: Algorithms 11 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Powering a number

Problem: Compute an , where n ∈ N.

Naive algorithm: Θ(n).
Divide-and-conquer algorithm:
 n/2 n/2
a ·a if n is even
an = (n−1)/2 (n−1)/2
a ·a · a if n is odd

T (n) = T (n/2) + Θ(1) ⇒ T (n) = Θ(lg n) .

Mumit Khan Licensed under CSE 221: Algorithms 11 / 12

 Divide and Conquer design paradigm Introduction to Divide and Conquer strategy Example: Binary

Conclusion

Divide and Conquer is just one of several algorithm design

strategies.
Used by many of the commonly used algorithms
Binary search
Merge sort
Fast Fourier Transform (FFT)
Finding closest pair of points
Matrix multiplication (Strassen’s algorithm)
Matrix inversion
Quicksort and (k th ) selection
...
Can be easily analyzed using recurrences
Often leads to efficient algorithms

Mumit Khan Licensed under CSE 221: Algorithms 12 / 12