Algorithm Design Patterns and Anti-Patterns

Algorithm design patterns. Ex.

Chapter 8 


Greedy.
Divide-and-conquer.
O(n log n) interval scheduling.
O(n log n) FFT.
 Dynamic programming. O(n2) edit distance.
NP and Computational  Duality. O(n3) bipartite matching.
Reductions.
Intractability


 Local search.
 Randomization.

Algorithm design anti-patterns.

 NP-completeness. O(nk) algorithm unlikely.
 PSPACE-completeness. O(nk) certification algorithm unlikely.
Slides by Kevin Wayne.
Copyright © 2005 Pearson-Addison Wesley.
All rights reserved.
 Undecidability. No algorithm possible.

1 2

Classify Problems According to Computational Requirements

8.1 Polynomial-Time Reductions Q. Which problems will we be able to solve in practice?

A working definition. [von Neumann 1953, Godel 1956, Cobham 1964, Edmonds 1965, Rabin 1966]
Those with polynomial-time algorithms.

Yes Probably no
Shortest path Longest path
Matching 3D-matching
Min cut Max cut
2-SAT 3-SAT
Planar 4-color Planar 3-color
Bipartite vertex cover Vertex cover

Primality testing Factoring

4
 Classify Problems Polynomial-Time Reduction

Desiderata. Classify problems according to those that can be solved in Desiderata'. Suppose we could solve X in polynomial-time. What else
polynomial-time and those that cannot. could we solve in polynomial time?
don't confuse with reduces from

Provably requires exponential-time. Reduction. Problem X polynomial reduces to problem Y if arbitrary

 Given a Turing machine, does it halt in at most k steps? instances of problem X can be solved using:
 Given a board position in an n-by-n generalization of chess,  Polynomial number of standard computational steps, plus
can black guarantee a win?  Polynomial number of calls to oracle that solves problem Y.

computational model supplemented by special piece

Frustrating news. Huge number of fundamental problems have defied Notation. X ≤ P Y. of hardware that solves instances of Y in a single step
classification for decades.
Remarks.
This chapter. Show that these fundamental problems are We pay for time to write down instances sent to black box ⇒
"computationally equivalent" and appear to be different manifestations instances of Y must be of polynomial size.
of one really hard problem. Note: Cook reducibility.
in contrast to Karp reductions

5 6

Polynomial-Time Reduction

Purpose. Classify problems according to relative difficulty.

Reduction By Simple Equivalence

Design algorithms. If X ≤ P Y and Y can be solved in polynomial-time,

then X can also be solved in polynomial time.
Basic reduction strategies.
Establish intractability. If X ≤ P Y and X cannot be solved in  Reduction by simple equivalence.
polynomial-time, then Y cannot be solved in polynomial time.  Reduction from special case to general case.
 Reduction by encoding with gadgets.
Establish equivalence. If X ≤ P Y and Y ≤ P X, we use notation X ≡ P Y.

up to cost of reduction

7
 Independent Set Vertex Cover

INDEPENDENT SET: Given a graph G = (V, E) and an integer k, is there a VERTEX COVER: Given a graph G = (V, E) and an integer k, is there a
subset of vertices S ⊆ V such that |S| ≥ k, and for each edge at most subset of vertices S ⊆ V such that |S| ≤ k, and for each edge, at least
one of its endpoints is in S? one of its endpoints is in S?

Ex. Is there an independent set of size ≥ 6? Yes. Ex. Is there a vertex cover of size ≤ 4? Yes.
Ex. Is there an independent set of size ≥ 7? No. Ex. Is there a vertex cover of size ≤ 3? No.

independent set

vertex cover

9 10

Vertex Cover and Independent Set Vertex Cover and Independent Set

Claim. VERTEX-COVER ≡P INDEPENDENT-SET. Claim. VERTEX-COVER ≡P INDEPENDENT-SET.

Pf. We show S is an independent set iff V − S is a vertex cover. Pf. We show S is an independent set iff V − S is a vertex cover.

⇒
 Let S be any independent set.
 Consider an arbitrary edge (u, v).
 S independent ⇒ u ∉ S or v ∉ S ⇒ u ∈ V − S or v ∈ V − S.
 Thus, V − S covers (u, v).

⇐
 Let V − S be any vertex cover.
 Consider two nodes u ∈ S and v ∈ S.
Observe that (u, v) ∉ E since V − S is a vertex cover.
independent set


 Thus, no two nodes in S are joined by an edge ⇒ S independent set. ▪

vertex cover

11 12
 Set Cover

SET COVER: Given a set U of elements, a collection S1, S2, . . . , Sm of

Reduction from Special Case to General Case subsets of U, and an integer k, does there exist a collection of ≤ k of
these sets whose union is equal to U?

Sample application.
m available pieces of software.
Basic reduction strategies.
Set U of n capabilities that we would like our system to have.
 Reduction by simple equivalence.
The ith piece of software provides the set Si ⊆ U of capabilities.
 Reduction from special case to general case.
Goal: achieve all n capabilities using fewest pieces of software.
 Reduction by encoding with gadgets.

Ex:
U = { 1, 2, 3, 4, 5, 6, 7 }
k=2
S1 = {3, 7} S4 = {2, 4}
S2 = {3, 4, 5, 6} S5 = {5}
S3 = {1} S6 = {1, 2, 6, 7}

14

Vertex Cover Reduces to Set Cover Polynomial-Time Reduction

Claim. VERTEX-COVER ≤ P SET-COVER. Basic strategies.

Pf. Given a VERTEX-COVER instance G = (V, E), k, we construct a set  Reduction by simple equivalence.
cover instance whose size equals the size of the vertex cover instance.  Reduction from special case to general case.
 Reduction by encoding with gadgets.
Construction.
 Create SET-COVER instance:
– k = k, U = E, Sv = {e ∈ E : e incident to v }
 Set-cover of size ≤ k iff vertex cover of size ≤ k. ▪

VERTEX COVER SET COVER

a b

e7 U = { 1, 2, 3, 4, 5, 6, 7 }
e2 e3 e4
k=2
Sa = {3, 7} Sb = {2, 4}
f e6 c
Sc = {3, 4, 5, 6} Sd = {5}
e1 e5 Se = {1} Sf= {1, 2, 6, 7}
k=2
e d

15 16
 Satisfiability

8.2 Reductions via "Gadgets" Literal: A Boolean variable or its negation. xi or xi

Clause: A disjunction of literals. C j = x1 " x2 " x3

!
Conjunctive normal form: A propositional " = C1 # C2 # C3 # C4
Basic reduction strategies.
formula Φ that is the conjunction of clauses.
!
 Reduction by simple equivalence.
 Reduction from special case to general case. !
SAT: Given CNF formula Φ, does it have a satisfying truth assignment?
 Reduction via "gadgets."

3-SAT: SAT where each clause contains exactly 3 literals.

each corresponds to a different variable

Ex: ( x1 " x2 " x3 ) # ( x1 " x2 " x 3 ) # ( x2 " x3 ) # ( x1 " x2 " x3 )

Yes: x1 = true, x2 = true x3 = false.

! 18

3 Satisfiability Reduces to Independent Set 3 Satisfiability Reduces to Independent Set

Claim. 3-SAT ≤ P INDEPENDENT-SET. Claim. G contains independent set of size k = |Φ| iff Φ is satisfiable.
Pf. Given an instance Φ of 3-SAT, we construct an instance (G, k) of
INDEPENDENT-SET that has an independent set of size k iff Φ is Pf. ⇒ Let S be independent set of size k.
satisfiable. S must contain exactly one vertex in each triangle.
Set these literals to true. and any other variables in a consistent way

Construction. Truth assignment is consistent and all clauses are satisfied.

G contains 3 vertices for each clause, one for each literal.
Connect 3 literals in a clause in a triangle. Pf ⇐ Given satisfying assignment, select one true literal from each
Connect literal to each of its negations. triangle. This is an independent set of size k. ▪

x1 x2 x1 x1 x2 x1

G G

! ! ! ! ! !
x2 x3 x1 x3 x2 x4 x2 x3 x1 x3 x2 x4

k=3 " = ( x1 # x2 # x 3 ) $ ( x1 # x2 # x 3 ) $ ( x1 # x2 # x 4 ) k=3 " = ( x1 # x2 # x 3 ) $ ( x1 # x2 # x 3 ) $ ( x1 # x2 # x 4 )

19 20
! ! ! ! ! ! ! ! ! ! ! !
 Review Self-Reducibility

Basic reduction strategies. Decision problem. Does there exist a vertex cover of size ≤ k?
 Simple equivalence: INDEPENDENT-SET ≡ P VERTEX-COVER. Search problem. Find vertex cover of minimum cardinality.
 Special case to general case: VERTEX-COVER ≤ P SET-COVER.
 Encoding with gadgets: 3-SAT ≤ P INDEPENDENT-SET. Self-reducibility. Search problem ≤ P decision version.
 Applies to all (NP-complete) problems in this chapter.
 Justifies our focus on decision problems.
Transitivity. If X ≤ P Y and Y ≤ P Z, then X ≤ P Z.
Pf idea. Compose the two algorithms.
Ex: to find min cardinality vertex cover.
Ex: 3-SAT ≤ P INDEPENDENT-SET ≤ P VERTEX-COVER ≤ P SET-COVER.  (Binary) search for cardinality k* of min vertex cover.
 Find a vertex v such that G − { v } has a vertex cover of size ≤ k* - 1.
– any vertex in any min vertex cover will have this property
 Include v in the vertex cover.
 Recursively find a min vertex cover in G − { v }.

delete v and all incident edges

21 22