Divide and Conquer Strategy

•Problem subdivision – Divide and Conquer:

•Binary search,
•Quick sort,
•Merge sort,
•Integer Arithmetic,
• Maximum sub-array,
•Master’s theorem and its uses. Greedy Strategy

•Greedy strategy:
•Principle, control abstraction, time analysis of
control abstraction,
• knapsack problem,
• Job scheduling algorithm,
•Single Source Shortest Path Dijkstra’s,
• Bellman-Ford
 Divide-and-Conquer

General idea:

Divide a problem into subprograms of the

same kind; solve subprograms using the
same approach, and combine partial
solution (if necessary).
 Searching in sorted arrays
• Suppose the array is arranged in increasing or
non-decreasing order
• Linear search on a sorted array still yields the
same analysis
– O( n ) worst-case time complexity
• Can exploit sorted structure by performing binary
search
• Strategy: inspect middle of the search list so that
half of the list is discarded at every step
 Algorithm 2: Binary Search
Algorithm BinarySearch(A,v):
Input: A SORTED array A of n numbers, search target v
Output: position i where A[i]=v, -1 if not found

low  0, high  n-1

while low <= high do
mid  (low+high)/2
if A[mid]= v then
return mid
else if A[mid] < v then
low  mid+1
else
high  mid-1
return -1
 Binary Search time complexity
• Time complexity analysis: how many
iterations of the while loop?
(assume worst-case)
• Observe values for low and high
– Note that before loop entry,
size of search space = n = high-low+1.
– On each suceeding iteration, search space is cut
in half
– Loop terminates when search space collapses to
0 or 1
 Binary Search time complexity
• search space size iteration number
n 1
n/2 2
n/4 3
… …
1 x
• x = number of iterations
• Observe 2x = n, log 2x = log n,
x = log n iterations carried out
• Binary Search worst-case time complexity:
O( log n )
 Binary Search time complexity

• Worst-case: O( log n )
– If v does not exist
• Best-case: O( 1 )
– When the target element v happens to be in the
middle of the array
• Average-case: O( log n )
– Technically, some statistical analysis needed
here (beyond the scope of this course)
 Binary Search (version 2)
• Can use recursion instead of iteration
• BinSearch(A,v):
return BinSearchHelper(A,0,n-1,v)

• BinSearchHelper(A,low,high,v):
if low > high then
return -1
mid  (low+high)/2
if A[mid] = v then
return mid
else if A[mid] < v then
return BinSearchHelper(A,mid+1,high,v)
else
return BinSearchHelper(A,low,mid-1,v)
 Mergesort
• Split array A[0..n-1] in two about equal halves and make
copies of each half in arrays B and C
• Sort arrays B and C recursively
• Merge sorted arrays B and C into array A as follows:
– Repeat the following until no elements remain in one of
the arrays:
• compare the first elements in the remaining
unprocessed portions of the arrays
• copy the smaller of the two into A, while incrementing
the index indicating the unprocessed portion of that
array
– Once all elements in one of the arrays are processed, copy
the remaining unprocessed elements from the other array
into A.
Pseudocode of Mergesort
Pseudocode of Merge
 Mergesort Example
8 3 2 9 7 1 5 4

8 3 2 9 7 1 5 4

8 3 2 9 71 5 4

8 3 2 9 7 1 5 4

3 8 2 9 1 7 4 5

2 3 8 9 1 4 5 7

1 2 3 4 5 7 8 9
 Analysis of Mergesort
• All cases have same efficiency: Θ(n log n)

• Number of comparisons in the worst case is close to

theoretical minimum for comparison-based sorting:

log2 n! ≈ n log2 n - 1.44n

• Space requirement: Θ(n) (not in-place)

• Can be implemented without recursion (bottom-up)

3. Merge Sort
The problem: 
Given a list of n numbers, sort them in non-decreasing order

idea: Split each part into 2 equal parts and sort each part using
Mergesort, then merge the tow sorted sub-lists into one sorted list.
 
The algorithm:
procedure MergeSort (low, high)
begin low  high
if
 low  high 
then mid   
2
call MergeSort (low, mid);
call MergeSort (mid+1, high);
call Merge (low, mid, high);
end;
procedure Merge (low, mid, high)

begin i  low, j  mid  1, k  low

while (i  mid and j  high)

if A(i) < A(j), then U(k) A(i);
i++;
else U(k) A(j);
j++;
k++;
if (i > mid)
move A(j) through A(high) to U(k) through U(high);

else
move A(i) through A(mid) to U(k) through U(high);
end;
Analysis:

Worst-Case for MergeSort is:

n
T (n)  2  T ( )  bn
2
 O(nLogn)
Average-Case? Merge sort pays no attention to
the original order of the list, it will keep divide the list
into half until sub-lists of length 1, then start merging.

Therefore Average-Case is the same as the Worst-Case.

 Quicksort
• Select a pivot (partitioning element) – here, the first
element
• Rearrange the list so that all the elements in the first
s positions are smaller than or equal to the pivot and
all the elements in the remaining n-s positions are
larger than or equal to the pivot (see next slide for an
algorithm)
p

A[i]p A[i]p

• Exchange the pivot with the last element in the first (i.e.,
) subarray — the pivot is now in its final position
• Sort the two subarrays recursively
 Analysis of Quicksort
• Best case: split in the middle — Θ(n log n)
• Worst case: sorted array! — Θ(n2)
• Average case: random arrays — Θ(n log n)

• Improvements:
– better pivot selection: median of three partitioning
– switch to insertion sort on small subfiles
– elimination of recursion
These combine to 20-25% improvement

• Considered the method of choice for internal sorting of

large files (n ≥ 10000)
4. Quick Sort

The problem:
 
Same as Merge Sort
 
Compared with Merge Sort:

• Quick Sort also use Divide-and-Conquer

strategy
• Quick Sort eliminates merging operations
How it works?
 
Use partition

Eg: Sort array 65, 70, 75, 80, 85, 60, 55, 50, 45
65 70 75 80 85 60 55 50 45

Pivot item

60 55 50 45 70 75 80 85
65

55 50 45 60 65 70 75 80 85

50 45 55 60 65 70 80 85

75
45 55 60 65 70 75 80 85
The algorithm:
procedure QuickSort (p, q)
begin
if p < q,
then call Partition (p, q, pivotposition);
call QuickSort (p, pivotposition –1);
call QuickSort (pivotposition+1, q);
end;

procedure Partition (low, high, pivotposition)

begin
v  A(low);
j  low;
for i  (low + 1) to high
if A(i) < v,
j++;
A(i)  A(j);
pivotposition  j;
A(low)  A(pivotposition);
end;
Analysis:

Worst-Case:
Call Partition O(n) times, each time takes
O(n) steps. So O(n2), and it is worse than
Merge Sort in the Worst-Case.
 
Best-Case:
Split the list evenly each time.
O(nLogn)
Average-Case:
Assume pivot is selected randomly, so the probability of
pivot being the kth element is equal, k.
1
prob ( pivot  k )  , k
n

C(n) = # of key comparison for n items

 (n  1)  C  (k  1)  C  (n  k )

• average it over all k, (CA(n) is average performance)

1 n
C A (n)  (n  1)   (C A (k  1)  C A (k  2))
n k 1

• multiply both side by n

n  C A ( n) 
n(n  1)  2  (C A (0)  C A (1)    C A (n  1)) (1)
• replace n by n-1
(n  1)  C A (n  1) 
(n  1)(n  2)  2  (C A (0)    C A (n  2)) (2)

• subtract (2) from (1)

n  C A (n)  (n  1)C A (n  1)  2  (n  1)  2  C A (n  1)
 n  C A (n)  2  (n  1)  (n  1)  C A (n  1)
• divide n(n + 1) for both sides
C A (n) C A (n  1) 2  (n  1)
 
n 1 n n(n  1)
C (n  2) 2  ( n  2) 2  (n  1)
 A  
n 1 (n  1)  n n(n  1)
C (n  3) 2  ( n  3) 2  (n  2) 2  (n  1)
 A   
n2 (n  2)  (n  1) ( n  1)  n n(n  1)

 Note : 
 C A (1)  0 
 
C (1) 1 2 n 1
 A  2(   )
2 2 3 3 4 n(n  1)
n
k 1
 0  2 
k 2 k ( k  1)
n n
1 1
 2    2   dk  2  ( Log e n  Log e 1)
k 2 k 1
k
 O( Logn)
 C A (n)  O (nLogn)

 Best-Case = Average-Case
Integer Arithmetic
1. Find the maximum and minimum

The problem: Given a list of unordered n elements,

find max and min

The straightforward algorithm:

 
max ← min ← A (1);
for i ← 2 to n do
if A (i) > max, max ← A (i);
if A (i) < min, min ← A (i);
 
Key comparisons: 2(n – 1)
List List 1 List 2
n elements n/2 elements n/2 elements

min, max min1, max1 min2, max2

min = MIN ( min1, min2 )

max = MAX ( max1, max2)
The Divide-and-Conquer algorithm:
procedure Rmaxmin (i, j, fmax, fmin); // i, j are index #, fmax,
begin // fmin are output parameters
case:
i = j: fmax ← fmin ← A (i);
i = j –1: if A (i) < A (j) then fmax ← A (j);
fmin ← A (i);
else fmax ← A (i);
fmin ← A (j);

else: mid ←  i  j  ;
 2 
call Rmaxmin (i, mid, gmax, gmin);
call Rmaxmin (mid+1, j, hmax, hmin);
fmax ← MAX (gmax, hmax);
fmin ← MIN (gmin, hmin);
end
end;
Eg: find max and min in the array:
22, 13, -5, -8, 15, 60, 17, 31, 47 ( n = 9 )

Index: 1 2 3 4 5 6 7 8 9
Array: 22 13 -5 -8 15 60 17 31 47

(1)
Rmaxmin(1, 9, 60, -8)
(6)

1, 5, 22, -8 6, 9, 60, 17
(2) (5) (7) (8)

1, 3, 22, -5 4, 5, 15, -8 6,7, 60, 17 8, 9, 47, 31

(3) (4)

1,2, 22, 13 3, 3,-5, -5

Analysis: For algorithm containing recursive calls, we can use
recurrence relation to find its complexity
T(n) - # of comparisons needed for Rmaxmin
Recurrence relation:

T (n)  0 n 1

T (n)  1 n2
 n
T (n)  2T ( )  2 otherwise
 2
n
T (n)  2T ( )  2
2
n n
 2  (2T ( )  2)  2  2 2  T ( 2 )  2 2  2
4 2
n n
 2 2  (2T ( )  2)  2 2  2  2 3  T ( 3 )  2 3  2 2  2
8 2

Assume n = 2k for some integer k

k 1 n k 1 k 2 1
2 T( k 1
)  (2 2  2 )
2
k 1 n k
 2  T (2)  (2  2)   1  n  2
2
 1.5n  2
 They don’t have to be the same constant.
We make them the same here for simplicity.
It will not affect the overall result.
Theorem:
b n 1
 where are a, b, c constants
T ( n)   n
 aT ( )  bn n 1
c

Claim:
O (n) ac

T (n)  O (nLogcn) ac
 Log c a
O (n ) ac
Proof: Assume n = ck
n
T (n)  a  T ( )  bn
c
n n n n
T (n)  a  (a  T ( 2 )  b  )  bn  a 2  T ( 2 )  ab   bn
c c c c
n n n
 a 2  (a  T ( 3 )  b  2 )  ab   bn
c c c
3 n a2 a
 a  T ( 3 )  bn  ( 2   1)
c c c

k n a k 1 a k 2 a0
 a  T ( k )  bn  ( k 1  k 2    0 )
c c c c
a a a
 a k  b  bn  (( ) k 1  ( ) k 2    ( ) 0 )
c c c
n a a a
 a k  b  k  bn  (( ) k 1  ( ) k 2    ( ) 0 )
c c c c
a a a a
 bn  ( ) k  bn  (( ) k1  ( ) k 2    ( ) 0 )
c c c c
k a
 bn  ( ( ) i )
i 0 c
 k
a i
If a < c,  ( c ) is a constant
i 0

T(n) = bn · a constant
 T(n) = O(n)
k
a
if a = c,  ( ) i  k  1
i 0 c

k
T (n)  bn1  bn  (k  1)
i 0

 bn  ( Log c n  1)
 O(nLogn)
 a k 1
( ) 1
k
a i c
If a > c,  (
c
) 
a
i 0
( ) 1
c
a k 1
k ( ) 1
a i
T ( n)  bn   ( )  bn  c
i 0 c
a
( ) 1
c
a k
 bn  ( )  b  a k  b  a Log c n
c
 b  n Log c a
Log c a
 O(n )
2. Integer multiplication

The problem:
 
Multiply two large integers (n digits)
 
The traditional way:
 
Use two for loops, it takes operations
The Divide-and-Conquer way:
 
Suppose large integers, , divide into two part a and
b, same as into c and d.
n
x: a b  x  a  10  b
2

y: c d  y  c  10  d
2

n n
x  y  (a  10  b)(c  10  d )
2 2

n
 ac  10 n  bd  10 (ad  bc)
2

So, transform the problem of multiply two integers of n-digit

n
into four subproblems of multiply two integers of -digit
2
Worst-Case is:
  n
T (n)  4  T ( )  bn
2
   O(n Log 2 4 )  O (n 2 )
however, it is same as the traditional way.


Therefore, we need to improve equation show before:
n
 ac  10 n  bd  10 2  (ad  bc)
n
 ac  10 n  bd  10 2  ((a  b)(c  d )  ac  bd )
Worst-Case is:
n
T (n)  3  T ( )  bn
2
 O(n Log 2 3 )  O(n1.58 )
The algorithm by Divide-and-Conquer:
Large-int function multiplication (x,y)
begin
n = MAX ( # of digits in x, #of digits in y);
if (x = 0) or (y = 0), return 0;
else if (n = 1), return x*y in the usual way;
else n
m =  2  ;
a = x divide 10m;
b = x rem 10m;
c = y divide 10m;
d = y rem 10m;
p1= MULTIPLICATION (a, c);
p2= MULTIPLICATION (b, d);
p3 = MULTIPLICATION
2m m
(a+b, c+d);
p1  10  p 2  10 ( p3  p1  p 2 )
return ;
end;
example: x = 143, y = 256

x = 143 n=3 m=1 a = 14 P=36608

P p1= p’
y = 256 b=3
p2= 18
c = 25
d=6
p3= p’’

x = 14 n=2 m=1 a=1 P’=350

P’ p1 = 2
y = 25 b=4
p2 = 20
c=2
d=5
p3 = 35

x = 17 n=2 m=1 a=1 P’’=527

P’’ p1 = 3
y = 31 b=7
p2 = 7
c=3
d=1
p3 = 32
Maximum Subarray Problem
 Maximum Subarray Problem
• You can buy a unit of stock, only one time, then sell it at a
later date
– Buy/sell at end of day

• Strategy: buy low, sell high

– The lowest price may appear after the highest price

• Assume you know future prices

• Can you maximize profit by buying at lowest price and

selling at highest price?
Buy lowest sell highest
 Brute force
• How many buy/sell pairs are possible over
n days?
• Evaluate each pair and keep track of
maximum
• Can we do better?
 Transformation
• Find sequence of days so that:
– the net change from last to first is maximized
• Look at the daily change in price
– Change on day i: price day i minus price day i-1
– We now have an array of changes (numbers), e.g.
12,-3,-24,20,-3,-16,-23,18,20,-7,12,-5,-22,14,-4,6
– Find contiguous subarray with largest sum
• maximum subarray
– E.g.: buy after day 7, sell after day 11
 Brute force again
• Trivial if only positive numbers (assume
not)

• Need to check O(n2) pairs

• For each pair, find the sum

• Thus total time is …

 Divide-and-Conquer
• A[low..high]
• Divide in the middle:
– A[low,mid], A[mid+1,high]
• Any subarray A[i,..j] is
(1) Entirely in A[low,mid]
(2) Entirely in A[mid+1,high]
(3) In both
• (1) and (2) can be found recursively
 Divide-and-Conquer (cont.)
• (3) find maximum subarray that crosses
midpoint
– Need to find maximum subarrays of the form
A[i..mid], A[mid+1..j], low <= i, j <= high
• Take subarray with largest sum of (1), (2),
(3)
 Divide-and-Conquer (cont.)
Find-Max-Cross-Subarray(A,low,mid,high)
left-sum = -∞
sum = 0
for i = mid downto low
sum = sum + A[i]
if sum > left-sum then
left-sum = sum
max-left = i
right-sum = -∞
sum = 0
for j = mid+1 to high
sum = sum + A[j]
if sum > right-sum then
right-sum = sum
max-right = j
return (max-left, max-right, left-sum + right-sum)
 Time analysis
• Find-Max-Cross-Subarray: O(n) time

• Two recursive calls on input size n/2

• Thus:
T(n) = 2T(n/2) + O(n)
T(n) = O(n log n)
 Divide and Conquer Strategy
•Problem subdivision – Divide and Conquer:
•Binary search,
•Quick sort,
•Merge sort,
•Integer Arithmetic,
• Maximum sub-array,
•Master’s theorem and its uses. Greedy Strategy

•Greedy strategy:
•Principle, control abstraction, time analysis of
control abstraction,
• knapsack problem,
• Job scheduling algorithm,
•Single Source Shortest Path Dijkstra’s,
• Bellman-Ford
 Fractional Knapsack Problem
• Knapsack capacity: W

• There are n items: the i-th item has value Pi and weight wi

• Goal:

– find xi such that for all 0  xi  1, i = 1, 2, .., n

 wixi  W and

 xiPi is maximum
 Fractional Knapsack - Example

• E.g.: 20
---
$80
Item 3 30 +

Item 2 50 50
20 $100
Item 1 30
20 +
10 10 $60

$60 $100 $120 $240

$6/pound $5/pound $4/pound

 Fractional Knapsack Problem
• Greedy strategy 1:
– Pick the item with the maximum value
• E.g.:
– W=1
– w1 = 100, P1 = 2
– w2 = 1, P2 = 1
– Taking from the item with the maximum value:
Total value taken = P1/w1 = 2/100
– Smaller than what the thief can take if choosing the
other item
Total value (choose item 2) = P2/w2 = 1
 Fractional Knapsack Problem
Greedy strategy 2:

• Pick the item with the maximum value per pound Pi/wi

• If the supply of that element is exhausted and the thief can carry
more: take as much as possible from the item with the next greatest
value per pound
• It is good to order items based on their value per pound
P1 P2 Pn
  ... 
w1 w2 wn
 Fractional Knapsack Problem
Alg.: Fractional-Knapsack (W, P[n], w[n])

1. While w > 0 and as long as there are items remaining

2. pick item with maximum Pi/wi

3. xi  min (1, w/wi)

4. remove item i from list

5. w  w – x iw i

• w – the amount of space remaining in the knapsack (w = W)

M=size of knapsack
 JOB SEQUENCING WITH
DEADLINES
The problem is stated as below.
• There are n jobs to be processed on a machine.
• Each job i has a deadline di≥ 0 and profit pi≥0 .
• Pi is earned iff the job is completed by its deadline.
• The job is completed if it is processed on a machine
for unit time.
• Only one machine is available for processing jobs.
• Only one job is processed at a time on the machine.
66
 JOB SEQUENCING WITH
DEADLINES (Contd..)
• A feasible solution is a subset of jobs J such that
each job is completed by its deadline.
• An optimal solution is a feasible solution with
maximum profit value.
Example : Let n = 4, (p1,p2,p3,p4) = (100,10,15,27),
(d1,d2,d3,d4) = (2,1,2,1)

67
 JOB SEQUENCING WITH
DEADLINES (Contd..)
Sr.No. Feasible Processing Profit value
Solution Sequence
(i) (1,2) (2,1) 110
(ii) (1,3) (1,3) or (3,1) 115
(iii) (1,4) (4,1) 127 is the optimal one
(iv) (2,3) (2,3) 25
(v) (3,4) (4,3) 42
(vi) (1) (1) 100
(vii) (2) (2) 10
(viii) (3) (3) 15
(ix) (4) (4) 27

68
 GREEDY ALGORITHM TO
OBTAIN AN OPTIMAL SOLUTION
• Consider the jobs in the non increasing
order of profits subject to the constraint that
the resulting job sequence J is a feasible
solution.
• In the example considered before, the non-
increasing profit vector is
(100 27 15 10) (2 1 2 1)
p1 p4 p3 p2 d1 d4 d3 d2
69
GREEDY ALGORITHM TO OBTAIN AN
OPTIMAL SOLUTION (Contd..)

J = { 1} is a feasible one
J = { 1, 4} is a feasible one with processing
sequence ( 4,1)
J = { 1, 3, 4} is not feasible
J = { 1, 2, 4} is not feasible
J = { 1, 4} is optimal

70