CS315: Design and Analysis

of Algorithms
Semester I, 2020/21

10: Divide and Conquer Algorithms

Dr. Ruwan Nawarathna

Department of Statistics & Computer Science
Faculty of Science
University of Peradeniya
Contents
• Algorithm design paradigms
• Divide and conquer paradigm
• Common divide and conquer algorithms
• Maximum sum subarray problem

2
Algorithm design paradigms
• They provide templates suited to solving a broad range of diverse
problems.

• They can be translated into common control and data structures

provided by most high-level languages.

• The temporal and spatial requirements of the algorithms which result

can be precisely analysed.

3
Algorithm design paradigms
• Brute force or exhaustive
• Divide and Conquer
• Dynamic Programming
• Greedy
• Branch and Bound
• Backtracking

4
Divide and Conquer Paradigm
• In computer science, divide and conquer is an algorithm design
paradigm based on multi-branched recursion.

• A divide and conquer algorithm works by recursively breaking down a

problem into two or more sub-problems of the same or related type,
until these become simple enough to be solved directly.

5
Divide and Conquer Paradigm
• Divide and Conquer algorithm solves a problem using following three
steps.
• Divide the problem into number of subproblems that are smaller instances of
the same problem

• Conquer the subproblems by solving them recursively

• If the subproblems are small enough just solve the subproblem in a straightforward
manner

• Combine the solutions to the subproblems into the solution for the original
problem

6
Well-known Divide and Conquer Algorithms
• Sorting (merge sort and quick sort) – already covered
• Binary search
• Matrix multiplication
• Straightforward
• Strassen’s method77
• Maximum subarray problem
• The Karatsuba algorithm is a fast multiplication algorithm

7
Maximum subarray

8
Maximum Subarray Problem
• Problem:
• Input: an array A[1..n] of (positive/negative) numbers.
• Output:
• (1) Indices i and j such that the subarray A[i..j] has the greatest sum of any nonempty
contiguous subarray of A, and
• (2) the sum of the values in A[i..j].

9
Applications
• Given a series of stock prices. Pick when to buy and when to gain max
profit.

• In bitmap images to detect the highest score subsequence which

represents the brightest area in an image

• Genomic sequence analysis employs maximum subarray algorithms

to identify important biological segments of protein sequences

10
Examples – Stock Prices

11
Brute-force Solution
• Check the sum of all subarrays
• Total number of subarrays

• Check all subarrays: O(n2)

12
Brute-force Solution
• Input: [-1,3,-2,5]
• Output: [3,-2,5] (sum: 6)

• Subarrays: [-1], [3], [-2], [5], [-1,3], [3,-2], [-2,5], [-1,3,-2], [3,-2,5], [-
1,3,-2,5]

• Optimal choice: [3,-2,5] (sum: 6)

13
Maximum subarray
• Can we do it in a divide-and-conquer manner?
• Yes, we can
Divide and Conquer (DC) Solution
• Divide: A[low..high] into two subarrays of as equal size as possible
by finding the midpoint mid

• Conquer:
• finding maximum subarrays of A[low..mid] and A[mid + 1..high]
• finding a max-subarray that crosses the midpoint

• Combine: returning the max of the three

• This strategy works because any subarray must either lie entirely
in one side of midpoint or cross the midpoint.

15
DC Solution
• Possible locations of a maximum subarray A[i..j] of A[low..high],
where mid = (low+high)/2
• entirely in A[low..mid] (low  i  j  mid)
• entirely in A[mid+1..high] (mid < i  j  high)
• crossing the midpoint (low  i  mid < j  high)
crossing the midpoint

low mid high

mid +1

entirely in A[low..mid] entirely in A[mid+1..high]

Possible locations of subarrays of A[low..high]

DC Solution – Finding the Crossing Max Subarray

Maximum
subarray:
Array between
max-left (2) and
max-right (9)
{5,3,-4,2,-3,7,1,2}
Sum = 13

17
DC 5 3 -4 2 -3 7 1 2
Solution -10 5 3 -4 2 -3 7 1 2 -5
Complete
Algorithm
5 3 7 1 2
-10 5 3 -4 2 -3 7 1 2 -5

5 3 2 7 1 2
-10 5 3 -4 2 -3 7 1 2 -5

5 3 -4 2 7 1 2 -5
-10 5 3 -4 2 -3 7 1 2 -5

-10 5 -3 7
-10 5 -3 7 18
DC maximum-subarray – Python Code
#Function to calculate the maximum crossing #Implementation of maximum subarray sum function
sum def maxSubarraySum(arr,low,high):
def maxCrossingSum(arr,low,mid,high): if low==high:
leftSum=-1000000 return arr[low]
rightSum=-1000000
mid=(low+high)//2
sum=0
for i in range(mid,low-1,-1): Left_MSS=maxSubarraySum(arr,low,mid)
sum+=arr[i] Right_MSS=maxSubarraySum(arr,mid+1,high)
if sum>leftSum: Crossing_Sum=maxCrossingSum(arr,low,mid,high)
leftSum=sum
return max(Left_MSS, Right_MSS, Crossing_Sum)
sum=0
for i in range(mid+1,high+1):
sum+=arr[i]
if sum>rightSum:
rightSum=sum

return leftSum+rightSum

19
Analysis – O(nlogn)
• No. of steps to find the maximum sum crossing sub array is O(n)
• See the code of MaxCrossingSubarray(A,low,mid,high)

• Maximum no. of divisions = O(logn)

• Time complexity: O(n)xO(logn) = O(nlogn)

20
O(n) Algorithm for Maximum Subarray
• Kadane's algorithm (Dynamic Programming)
• Will be discussed under «Dynamic Programming» section

21
Binary Search

22
Problem : Search
• We are given a list of records.
• Each record has an associated key.
• Give efficient algorithm for searching for a record containing a
particular key.
• Efficiency is quantified in terms of average time analysis (number of
comparisons) to retrieve an item.

23
Search
[0] [1] [2] [3] [4] [ 700 ]

Number 701466868
Number 281942902 Number 233667136 Number 580625685 Number 506643548

… Number 155778322

• Each record in list has an associated key. Number 580625685

• In this example, the keys are ID numbers.

• Given a particular key, how can we efficiently retrieve

the record from the list?

24
Linear(Serial) Search
• Step through array of records, one at a time.
• Look for record with matching key.
• Search stops when
• record with matching key is found
• or when search has examined all records without success.

25
Linear Search Analysis
• Worst Case
• Consider cases where we must loop over all n records:
• desired record appears in the last position of the array
• desired record does not appear in the array at all
• For an array of n elements, the worst case time for linear search requires n array
accesses: O(n).

• Best Case
• Search key is the first element of the array
• Best case = O(1)

• Average Case = O(n)

26
Binary Search
• Perhaps we can do better than O(n) in the average case?

• Assume that we are give an array of records that is sorted. For

instance:
• an array of records with integer keys sorted from smallest to largest (e.g., ID
numbers), or
• an array of records with string keys sorted in alphabetical order (e.g., names).

27
Binary Search

Example: sorted array of integer keys. Target=7.

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53

28
Relation to Binary Search Tree
Array of previous example:
3 6 7 11 32 33 53

Corresponding complete binary search tree

11
6 33

3 7 32 53

29
Search for target = 7
Find midpoint:
3 6 7 11 32 33 53

Start at root:

11
6 33

3 7 32 53

30
Search for target = 7
Search left subarray:
3 6 7 11 32 33 53

Search left subtree:

11
6 33

3 7 32 53

31
Search for target = 7
Find approximate midpoint of subarray:
3 6 7 11 32 33 53

Visit root of subtree:

11
6 33

3 7 32 53

32
Search for target = 7

Search right subarray:

3 6 7 11 32 33 53

Search right subtree:

11
6 33

3 7 32 53

33
Binary Search : Python Code
def binarySearch(array, x, low, high):
if high >= low:

mid = low + (high - low)//2

# If found at mid, then return it

if array[mid] == x:
return mid

# Search the left half

elif array[mid] > x:
return binarySearch(array, x, low, mid-1)

# Search the right half

else:
return binarySearch(array, x, mid + 1, high)

else:
return -1
34
Binary Search: Analysis
• Worst Case
• In the worst case, the item X does not exist in the array A at all.
• Each level in the recursion, we split the array in half (divide by two).
• Therefore, maximum recursion depth is O(logn) and each level only one operation
(comparison) is performed
• Hence, worst case = O(logn)

• Best case - O (1) comparisons

• In the best case, the item X is the middle in the array A. A constant number of
comparisons (actually just 1) are required.

• Average case = O(logn)

35