Divide and Conquer

1
 A Useful Recurrence Relation

Def. T(n) = number of comparisons to mergesort an input of size n.

Mergesort recurrence.

 0 if n 1

T(n)   T  n /2   T  n /2   n otherwise
 merging
 solve left half solve right half


Solution. T(n) = O(n log2 n).

2
 Proof by Recursion Tree

 0 if n 1

T(n)   2T(n /2)  n otherwise

 sorting both halves merging

 T(n) n

T(n/2) T(n/2) 2(n/2)

T(n/4) T(n/4) T(n/4) T(n/4) 4(n/4)

log2n
...
T(n / 2k) 2k (n / 2k)

...

T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) n/2 (2)

n log2n
3
5.3 Counting Inversions
 Counting Inversions

Music site tries to match your song preferences with others.

You rank n songs.
Music site consults database to find people with similar tastes.

Similarity metric: number of inversions between two rankings.

 My rank: 1, 2, …, n.
 Your rank: a1, a2, …, an.
 Songs i and j inverted if i < j, but ai > aj.

Songs
A B C D E
Inversions
Me 1 2 3 4 5
3-2, 4-2
You 1 3 4 2 5

Brute force: check all (n2) pairs i and j.

5
 Applications

Applications.
Voting theory.
Collaborative filtering.
Measuring the "sortedness" of an array.
Sensitivity analysis of Google's ranking function.
Rank aggregation for meta-searching on the Web.
Nonparametric statistics (e.g., Kendall's Tau distance).

6
 Counting Inversions: Divide-and-Conquer

Divide-and-conquer.

1 5 4 8 10 2 6 9 12 11 3 7

7
 Counting Inversions: Divide-and-Conquer

Divide-and-conquer.
 Divide: separate list into two pieces.

Divide: O(1).
1 5 4 8 10 2 6 9 12 11 3 7

1 5 4 8 10 2 6 9 12 11 3 7

8
 Counting Inversions: Divide-and-Conquer

Divide-and-conquer.
 Divide: separate list into two pieces.
 Conquer: recursively count inversions in each half.

Divide: O(1).
1 5 4 8 10 2 6 9 12 11 3 7

1 5 4 8 10 2 6 9 12 11 3 7 Conquer: 2T(n / 2)

5 blue-blue inversions 8 green-green inversions

5-4, 5-2, 4-2, 8-2, 10-2 6-3, 9-3, 9-7, 12-3, 12-7, 12-11, 11-3, 11-7

9
 Counting Inversions: Divide-and-Conquer

Divide-and-conquer.
 Divide: separate list into two pieces.
 Conquer: recursively count inversions in each half.
 Combine: count inversions where ai and aj are in different halves,
and return sum of three quantities.

Divide: O(1).
1 5 4 8 10 2 6 9 12 11 3 7

1 5 4 8 10 2 6 9 12 11 3 7 Conquer: 2T(n / 2)

5 blue-blue inversions 8 green-green inversions

9 blue-green inversions
Combine: ???
5-3, 4-3, 8-6, 8-3, 8-7, 10-6, 10-9, 10-3, 10-7

Total = 5 + 8 + 9 = 22.

10
 Counting Inversions: Combine

Combine: count blue-green inversions

Assume each half is sorted.
Count inversions where ai and aj are in different halves.
Merge two sorted halves into sorted whole.
to maintain sorted invariant

3 7 10 14 18 19 2 11 16 17 23 25
6 3 2 2 0 0

13 blue-green inversions: 6 + 3 + 2 + 2 + 0 + 0 Count: O(n)

2 3 7 10 11 14 16 17 18 19 23 25 Merge: O(n)

T(n)  T  n/2  T  n/2  O(n)  T(n)  O(nlog n)

11
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=6

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

auxiliary array

Total:

12
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=6

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6

2 auxiliary array

Total: 6

13
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=6

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6

2 auxiliary array

Total: 6

14
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=6

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6

2 3 auxiliary array

Total: 6

15
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=5

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6

2 3 auxiliary array

Total: 6

16
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=5

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6

2 3 7 auxiliary array

Total: 6

17
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=4

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6

2 3 7 auxiliary array

Total: 6

18
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=4

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6

2 3 7 10 auxiliary array

Total: 6

19
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=3

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6

2 3 7 10 auxiliary array

Total: 6

20
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=3

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6 3

2 3 7 10 11 auxiliary array

Total: 6 + 3

21
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=3

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6 3

2 3 7 10 11 auxiliary array

Total: 6 + 3

22
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=3

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6 3

2 3 7 10 11 14 auxiliary array

Total: 6 + 3

23
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=2

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6 3

2 3 7 10 11 14 auxiliary array

Total: 6 + 3

24
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=2

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6 3 2

2 3 7 10 11 14 16 auxiliary array

Total: 6 + 3 + 2

25
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=2

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6 3 2

2 3 7 10 11 14 16 auxiliary array

Total: 6 + 3 + 2

26
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=2

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6 3 2 2

2 3 7 10 11 14 16 17 auxiliary array

Total: 6 + 3 + 2 + 2

27
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=2

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6 3 2 2

2 3 7 10 11 14 16 17 auxiliary array

Total: 6 + 3 + 2 + 2

28
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=2

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6 3 2 2

2 3 7 10 11 14 16 17 18 auxiliary array

Total: 6 + 3 + 2 + 2

29
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=1

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6 3 2 2

2 3 7 10 11 14 16 17 18 auxiliary array

Total: 6 + 3 + 2 + 2

30
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=1

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6 3 2 2

2 3 7 10 11 14 16 17 18 19 auxiliary array

Total: 6 + 3 + 2 + 2

31
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

first half exhausted i=0

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6 3 2 2

2 3 7 10 11 14 16 17 18 19 auxiliary array

Total: 6 + 3 + 2 + 2

32
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=0

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6 3 2 2 0

2 3 7 10 11 14 16 17 18 19 23 auxiliary array

Total: 6 + 3 + 2 + 2 + 0

33
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=0

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6 3 2 2 0

2 3 7 10 11 14 16 17 18 19 23 auxiliary array

Total: 6 + 3 + 2 + 2 + 0

34
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=0

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6 3 2 2 0 0

2 3 7 10 11 14 16 17 18 19 23 25 auxiliary array

Total: 6 + 3 + 2 + 2 + 0 + 0

35
 Merge and Count

Merge and count step.

Given two sorted halves, count number of inversions where ai and aj
are in different halves.
Combine two sorted halves into sorted whole.

i=0

3 7 10 14 18 19 2 11 16 17 23 25 two sorted halves

6 3 2 2 0 0

2 3 7 10 11 14 16 17 18 19 23 25 auxiliary array

Total: 6 + 3 + 2 + 2 + 0 + 0 = 13

36
 Counting Inversions: Implementation

Pre-condition. [Merge-and-Count] A and B are sorted.

Post-condition. [Sort-and-Count] L is sorted.

Sort-and-Count(L) {
if list L has one element
return 0 and the list L

Divide the list into two halves A and B

(rA, A)  Sort-and-Count(A)
(rB, B)  Sort-and-Count(B)
(rB, L)  Merge-and-Count(A, B)

return r = rA + rB + r and the sorted list L

}

37
5.4 Closest Pair of Points
 Closest Pair of Points

Closest pair. Given n points in the plane, find a pair with smallest
Euclidean distance between them.

Fundamental geometric primitive.

 Graphics, computer vision, geographic information systems,
molecular modeling, air traffic control.
 Special case of nearest neighbor, Euclidean MST, Voronoi.
fast closest pair inspired fast algorithms for these problems

Brute force. Check all pairs of points p and q with (n2) comparisons.

1-D version. O(n log n) easy if points are on a line.

Assumption. No two points have same x coordinate.

to make presentation cleaner

39
Closest Pair of Points: First Attempt

40
 Closest Pair of Points: First Attempt

Divide. Sub-divide region into 4 quadrants.

41
 Closest Pair of Points: First Attempt

Divide. Sub-divide region into 4 quadrants.

Obstacle. Impossible to ensure n/4 points in each piece.

42
 Closest Pair of Points

Algorithm.
 Divide: draw vertical line L so that roughly ½n points on each side.

43
 Closest Pair of Points

Algorithm.
 Divide: draw vertical line L so that roughly ½n points on each side.
 Conquer: find closest pair in each side recursively.

21

12

44
 Closest Pair of Points

Algorithm.
 Divide: draw vertical line L so that roughly ½n points on each side.
 Conquer: find closest pair in each side recursively.
 Combine: find closest pair with one point in each side. seems like (n2)

 Return best of 3 solutions.

8
21

12

45
 Closest Pair of Points

Find closest pair with one point in each side, assuming that distance < .

21

 = min(12, 21)
12

46
 Closest Pair of Points

Find closest pair with one point in each side, assuming that distance < .
 Observation: only need to consider points within  of line L.

21

 = min(12, 21)
12

 47
 Closest Pair of Points

Find closest pair with one point in each side, assuming that distance < .
 Observation: only need to consider points within  of line L.
 Sort points in 2-strip by their y coordinate.

L
7

5
4 21

 = min(12, 21)
12 3

 48
 Closest Pair of Points

Find closest pair with one point in each side, assuming that distance < .
 Observation: only need to consider points within  of line L.
 Sort points in 2-strip by their y coordinate.
 Only check distances of those within 11 positions in sorted list!

L
7

5
4 21

 = min(12, 21)
12 3

 49
 Closest Pair of Points

Def. Let si be the point in the 2-strip, with

the ith smallest y-coordinate.

Claim. If |i – j|  12, then the distance between 39 j

si and sj is at least .
31
Pf.
 No two points lie in same ½-by-½ box.
 Two points at least 2 rows apart ½
have distance  2(½). ▪ 2 rows
29
30
½

i 27
28 ½
Fact. Still true if we replace 12 with 7.

26
25

  50
 Closest Pair Algorithm

Closest-Pair(p1, …, pn) {
Compute separation line L such that half the points O(n log n)
are on one side and half on the other side.

1 = Closest-Pair(left half)
2T(n / 2)
2 = Closest-Pair(right half)
 = min(1, 2)

Delete all points further than  from separation line L O(n)

Sort remaining points by y-coordinate. O(n log n)

Scan points in y-order and compare distance between

each point and next 11 neighbors. If any of these O(n)
distances is less than , update .

return .
}

51
 Closest Pair of Points: Analysis

Running time.

T(n)  2T n/2  O(n log n)  T(n)  O(n log 2 n)


Q. Can we achieve O(n log n)?

A. Yes. Don't sort points in strip from scratch each time.

 Each recursive returns two lists: all points sorted by y coordinate,
and all points sorted by x coordinate.
 Sort by merging two pre-sorted lists.

T(n)  2T n/2  O(n)  T(n)  O(n log n)


52
5.5 Integer Multiplication
 Integer Arithmetic

Add. Given two n-digit integers a and b, compute a + b.

O(n) bit operations.

Multiply. Given two n-digit integers a and b, compute a × b.

Brute force solution: (n2) bit operations.
1 1 0 1 0 1 0 1
* 0 1 1 1 1 1 0 1
1 1 0 1 0 1 0 1 0

Multiply
0 0 0 0 0 0 0 0 0
1 1 0 1 0 1 0 1 0
1 1 0 1 0 1 0 1 0
1 1 1 1 1 1 0 1 1 1 0 1 0 1 0 1 0
1 1 0 1 0 1 0 1 1 1 0 1 0 1 0 1 0
+ 0 1 1 1 1 1 0 1 1 1 0 1 0 1 0 1 0
1 0 1 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0
Add 0 1 1 0 1 0 0 0 0 0 0 0 0 0 0 1 0
54
 Divide-and-Conquer Multiplication: Warmup

To multiply two n-digit integers:

 Multiply four ½n-digit integers.
 Add two ½n-digit integers, and shift to obtain result.

x  2 n / 2  x1  x0
y  2 n / 2  y1  y0
xy  2 n/2
 x1  x0  2 n/2

 y1  y0  2 n  x1 y1  2 n / 2  x1 y0  x0 y1   x0 y0



T(n)  4T n /2   (n)  T(n)  (n 2 )

recursive calls add, shift

assumes n is a power of 2


55
 Karatsuba Multiplication

To multiply two n-digit integers:

 Add two ½n digit integers.
 Multiply three ½n-digit integers.
 Add, subtract, and shift ½n-digit integers to obtain result.

x  2 n / 2  x1  x0
y  2 n / 2  y1  y0
xy  2 n  x1 y1  2 n / 2  x1 y0  x0 y1   x0 y0
 2 n  x1 y1  2 n / 2   (x1  x0 ) (y1  y0 )  x1 y1  x0 y0   x0 y0
A B A C C

Theorem. [Karatsuba-Ofman, 1962] Can multiply two n-digit integers


in O(n1.585) bit operations.

T(n)  T  n /2   T  n /2   T 1 n /2   (n)

recursive calls add, subtract, shift

log 2 3
 T(n)  O(n )  O(n1.585 )

56
 Karatsuba: Recursion Tree

 0 if n  1 log 2 n
23
1 log 2 n
1
T(n)  
 3T(n /2)  n otherwise
T(n)   n 
3 k
2

3 1
 3nlog 2 3  2
k0 2

 T(n) n

T(n/2) T(n/2) T(n/2) 3(n/2)

T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) 9(n/4)

... ...

T(n / 2k) 3k (n / 2k)

... ...

T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) 3 lg n (2)

57