Administrivia, Lecture 5

HW #2 was assigned on Sunday, January 20. It is

due on Thursday, January 31.
Please use the correct edition of the textbook!
We have a quiz this Thursday (20 minutes) at the
beginning of lecture
The DQMaxMin problem is treated in CLRS 9.1
Grading will drop the single worst HW or quiz.
Selection
Recall: best pivot in QSort is the median
It is too expensive to find the median we use a random
element as the pivot instead
This leads to SELECTION:
Given a list L and a number k, Select (L, k) returns
the k
th
smallest element of L
e.g., when k = |L|/2 Select(L,k) returns the median
What is an efficient algorithm?
Sorting + finding k
th
smallest O(n log n) (can do better)
D/Q Recursion: use the pivot from QSort
Randomized-Select (CLRS 9.2, page 186)
if i<k, look in right part for (k-i)
th
smallest
if i>k, look in left part for k
th
smallest
Analysis of Randomized-Select (9.2)
Average-case time complexity





Inductive hypothesis: Want :

=
+ =
1
1
) ( }) , (max{
1
) (
n
k
n O k n k T
n
n T

=
+ =
1
2 /
) ( ) (
2
n
n k
n O k T
n
n c n T s ) (
cn n O k k
n
c
n O ck
n
n T
n
k
n
k
n
n k
s + |
.
|

\
|
s + =


=

=
) (
2
) (
2
) (
1
1
1 2 /
1
1
2 /
Need for induction step
Analysis of Randomized-Select (9.2, p. 188)
Want:
|
.
|

\
|
= + + s
+
|
.
|

\
|
+ = +
|
|
.
|

\
|
+ =
+
|
|
.
|

\
|
+

=
+
|
.
|

\
|

s
+ |
.
|

\
|
s + =


=

=
an
c cn
cn an
c cn
an
n
n
c an
n n
n
c
an
n n n n
n
c
an
n n n n
n
c
an k k
n
c
n O ck
n
n T
n
k
n
k
n
n k
2 4 2 4
3
2
2
1
4
3
2
2 4
3
2
) 2 2 / 3 4 / (
2
( 2
2
) 1 2 / )( 2 2 / (
2
) 1 ( 2
2
) (
2
) (
2
2 2
1
1
1 2 /
1
1
2 /
s cn cn/4 c/2 an > 0 n(c/4 a) > c/2
Choose c s.t. c/4 a > 0 c > 4a T(n) = O(n)
average-case
Randomized Selection
Worst case:
O(n
2
) as in QSort analysis
But: Suppose we can guarantee a good pivot,
e.g., such that n/4 s i s 3n/4
subproblem size s 3n/4
Let s(n) time to find good pivot
t(n) s s(n) + cn + t(3n/4)

s(n) = find pivot
cn = pivot, make subproblem
t(3n/4) = solve subproblem
Randomized Selection
Suppose further:
S(n) s dn for some d
t(n) s (c+d)n + t(3n/4)
Claim: t(n) s kn for some k would follow
Again, we use Constructive Induction (Substitution):
Induction Hypothesis: t(m) s km for m s n-1
Induction Step: t(n) s (c+d)n + k(3n/4)
= (c+d+3k/4)n
which we want to be equivalent to t(n) s kn
But this is true if k > 4(c+d)
Worst-Case Linear-Time Selection (9.3)
Of mostly theoretical interest
How to find a good median in linear time
divide into groups of 5 c
1
n
find the median of each group c
2
n/5
find the median of medians, recursively T(n/5)
Pivoting around this element gives at worst 3:7 split
Worst-Case Linear-Time Selection
Blum-Floyd-Pratt-Rivest-Tarjan, 1973
divide L into groups of 5 c
1
n
find the median of each group c
2
n/5
Let S = array of medians of each group c
3
n
find median of medians (i.e., median of S) recursively;
use as pivot t(n/5)

Total work: t(n) s kn + t(n/5) + t(7n/10 + d)
In general: a(n) s kn + a(e
1
n) + a(e
2
n) + a(e
3
n),
with k, e
1
, e
2
, e
3
constants s.t. e
1
+ e
2
+ e
3
< 1,
then a(n) s c*n for some c* t(n) = O(n)
More Careful Exposition
Recursive call has size s 7n/10 + 6 (CLRS p. 191)
T(n) e u(1), n s M (M = 140)
T(n) s T(n/5) + T(7n/10 + 6) + O(n), n > M
Prove linearity by substitution
Assume T(n) s cn for some constant c, and for all n s M
T(n) s cn/5 + c(7n/10 + 6) + an
s cn/5 + c + 7cn/10 + 6c + an
= 9cn/10 + 7c + an
= cn + (-cn/10 + 7c + an)
The induction step goes through if cn/10 + 7c + an s 0
pick c large enough so that c(n/10 - 7) > an n > M
D/Q for Arithmetic
Multiplying Large Integers (Karatsuba-Ofman)
A = a
0
+ r
1
a
1
+ ... + r
n-1
a
n-1
, r radix
classic approach O(n
2
) work
Can we apply D/Q?
Let n = 2s, r = 10 radix
AB = xz + 10
s
(wz + xy) + 10
2s
wy
T(n) = 4T(n/2) + u(n) a = 4, b = 2 in Master Method
T(n) e O(n
2
)
Need to reduce # subproblems, i.e., want a < 4
Observation: r = (w+x)(y+z) = wy + (wz+xy) + xz
r (w+x)(y+z)
p wy
q xz
return 10
2s
p + 10
s
(r-p-q) + q
T(n) e O(n
log
2
3
) = O(n
1.59
)
Matrix Multiplication
A = [] , B = [] are n x n matrices
a
11
, a
12
, etc are n/2 x n/2 submatrices
M = AB = []
where m
11
= a
11
b
11
+ a
12
b
21
etc.
Evaluation requires 8 multiplies, 4 adds
T(n) = 8T(n/2) + O(n) e O(n
3
)
Strassens Matrix Multiplication
p
1
= (a
21
+ a
22
- a
11
)(b
22
- b
12
+ b
11
)
p
2
= a
11
b
11
p
3
= a
12
b
21
p
4
= (a
11
-a
21
)(b
22
-b
12
)
p
5
= (a
21
+a
22
)(b
12
- b
11
)
p
6
= (a
12
-a
21
+a
11
-a
22
)b
22
p
7
= a
22
(b
11
+b
22
-b
12
-b
21
)
AB
11
= p
2
+ p
3
AB
12
= p
1
+ p
2
+ p
5
+ p
6

AB
21
= p
1
+ p
2
+ p
4
+ p
7
AB
22
= p
1
+ p
2
+ p
4
+ p
5
T(n) e O(n
2.81
) // 7 multiplies, 24 adds
Can get to 15 adds

Break
Question: Given the adjacency matrix of an
undirected graph, describe a fast algorithm that
finds all triangles in the graph.
(Why am I asking this now?)
A Computational Geometry Excursion
Given simple polygon P, is x in P? u(n) ?
Given n points in the plane, draw a simple n-gon
through them
u(n log n) ?
Given n points in the plane, return their convex hull
u(n log n) ?
Does it matter whether we must return vertices or return the polygon?
Given n points in the plane, return the closest pair
u(n log n) ?
Given n h-, v- segments, return all intersections
O(f(n) + g(k)), where k = output size?
plane-sweep inductive paradigm

DQ Closest Pair
Nave O(n
2
)
D/Q Approach: O(n log n)
(1) Split into two pointsets S
1
, S
2

by x-coord natural order
(2) Find closest pair distances d
1
, d
2
in S
1
, S
2
respectively
without loss of generality, can assume d
1
<d
2

(3) Merge the solutions
Do we have to check all s
1
-s
2
pairs, s
1
e S
1
, s
2
e S
2
?
What if there are lots of points in middle strip?
Key: Step (3)
Observation: There are at most O(1) points in middle strip with A y s d
1
DQ Closest Pair
(1) sort by x-coord tool: one-time sort O(n log n)
(2) solve subproblems of size n/2 2T(n/2)
(3) eliminate points outside strips O(n)
(4) sort by y-coord within strips O(n log n)
(5) compare each point to s 6 neighbors O(n)
Complexity
O(n log n) + T(n) = 2T(n/2) + O(n log n)
T(2) = 1
T(n) e O(n log
2
n)
Not what we wanted: what went wrong?
DQ Closest Pair
Why keep on sorting by y-coordinate?
(1) sort by x-coordinate, and sort by y-coordinate
Manber textbook: Tool: Strengthening the I.H.
Given <n points in the plane, we can find smallest
distance and output the set sorted by y-coordinate
N.B.: Udi Manber is Chief Scientist at Yahoo
Note: We exploited geometry
e.g., what is maximum possible degree of a vertex in the
minimum spanning tree?
If the problem is geometric?
If the problem is non-geometric?

Convex Hull
Goal: Return the hull points in order
What is a lower bound? Tool: reduction
Many approaches: see CLRS 33.3
Stretch u(n
2
)
Wrap u(n
2
)
Graham Scan u(n log n)
D/Q u(n log n)
left-right split
find convex hulls
merge convex hulls
Tool: Do you need to sort in order to bisect (split) ???
Yet Another Geometric Problem
Given two convex polygons, return intersection
Note 1: Intersection of (AB), (CD)
= Union of (AB), (AD), (BC), (BD)
I.e., intersection of unions is union of intersections
Note 2: How is a convex polygon represented?
Cyclic order
Note 3: How difficult is it to compute the
intersection of a 3-gon/4-gon, or 4-gon/4-gon?
O(1) time
Tool: Solving any bounded-size instance = O(1) time !
Can you get an O(n) algorithm?