Chapter 21

Reductions and NP

CS 473: Fundamental Algorithms, Spring 2011

April 14, 2011

21.1 Reductions Continued

21.1.0.1 Polynomial Time Reduction
A polynomial time reduction from a decision problem X to a decision problem Y is an
algorithm A that has the following properties:

• given an instance IX of X, A produces an instance IY of Y

• A runs in time polynomial in |IX |. This implies that |IY | (size of IY ) is polynomial in
|IX |

• Answer to IX YES iff answer to IY is YES.

Notation: X ≤P Y if X reduces to Y

Proposition 21.1.1 If X ≤P Y then a polynomial time algorithm for Y implies a polyno-

mial time algorithm for X.
Such a reduction is called a Karp reduction. Most reductions we will need are Karp
reductions

21.1.0.2 A More General Reduction

Turing Reduction (the one given in the book)

Problem X polynomial time reduces to Y if there is an algorithm A for X that has the
following properties:

1
 • on any given instance IX of X, A uses polynomial in |IX | “steps”

• a step is either a standard computation step or

• a sub-routine call to an algorithm that solves Y

Note: In making sub-routine call to algorithm to solve Y , A can only ask questions of
size polynomial in |IX |. Why?
Above reduction is called a Turing reduction.

21.1.0.3 Example of Turing Reduction

Input Collection of arcs on a circle.

Goal Compute the maximum number of non-overlapping arcs.

Reduced to the following problem:?

Input Collection of intervals on the line.

Goal Compute the maximum number of non-overlapping intervals.

How? Used algorithm for interval problem multiple times.

21.1.0.4 Turing vs Karp Reductions

• Turing reductions more general than Karp reductions

• Turing reduction useful in obtaining algorithms via reductions

• Karp reduction is simpler and easier to use to prove hardness of problems

• Perhaps surprisingly, Karp reductions, although limited, suffice for most known NP-
Completeness proofs

21.1.1 The Satisfiability Problem (SAT)

21.1.1.1 Propositional Formulas
Definition 21.1.2 Consider a set of boolean variables x1 , x2 , . . . xn
• A literal is either a boolean variable xi or its negation ¬xi

• A clause is a disjunction of literals. For example, x1 ∨ x2 ∨ ¬x4 is a clause

• A formula in conjunctive normal form (CNF) is propositional formula which is a

conjunction of clauses

– (x1 ∨ x2 ∨ ¬x4 ) ∧ (x2 ∨ ¬x3 ) ∧ x5 is a formula in CNF

2
 • A formula ϕ is in 3CNF if it is a CNF formula such that every clause has exactly 3
literals
– (x1 ∨x2 ∨¬x4 )∧(x2 ∨¬x3 ∨x1 ) is a 3CNF formula, but (x1 ∨x2 ∨¬x4 )∧(x2 ∨¬x3 )∧x5
is not.

21.1.1.2 Satisfiability

SAT
Given a CNF formula ϕ, is there a truth assignment to variables such that ϕ evaluates to
true?

Example 21.1.3 (x1 ∨ x2 ∨ ¬x4 ) ∧ (x2 ∨ ¬x3 ) ∧ x5 is satisfiable; take x1 , x2 , . . . x5 to be all

true
(x1 ∨ ¬x2 ) ∧ (¬x1 ∨ x2 ) ∧ (¬x1 ∨ ¬x2 ) ∧ (x1 ∨ x2 ) is not satisfiable

3-SAT
Given a 3CNF formula ϕ, is there a truth assignment to variables such that ϕ evaluates to
true?

21.1.1.3 Importance of SAT and 3-SAT

• SAT and 3-SAT are basic constraint satisfaction problems
• Many different problems can reduced to them because of the simple yet powerful ex-
pressivity of logical constraints
• Arise naturally in many applications involving hardware and software verification and
correctness
• As we will see, it is a fundamental problem in theory of NP-Completeness

21.1.2 Sat and 3-SAT

21.1.2.1 SAT ≤P 3-SAT
Easy to see that 3-SAT ≤P SAT. A 3-SAT instance is also an instance of SAT.

We can show that SAT ≤P 3-SAT.

Given ϕ a SAT formula we create a 3-SAT formula ϕ0 such that

• ϕ is satisfiable iff ϕ0 is satisfiable
• ϕ0 can be constructed from ϕ in time polynomial in |ϕ|.
Idea: if a clause of ϕ is not of length 3, replace it with several clauses of length exactly 3

3
21.1.2.2 SAT ≤P 3-SAT

Reduction Ideas
Challenge: Some of the clauses in ϕ may have less or more than 3 literals. For each clause
with < 3 or > 3 literals, we will construct a set of logically equivalent clauses.
• Case clause with 1 literal: Let c = `. Let u, v be new variables. Consider
c0 = (` ∨ u ∨ v) ∧ (` ∨ u ∨ ¬v) ∧ (` ∨ ¬u ∨ v) ∧ (` ∨ ¬u ∨ ¬v)
Observe that c0 is satisfiable iff c is satisfiable

21.1.2.3 SAT ≤P 3-SAT (contd)

Reduction Ideas: 2 and more literals

• Case clause with 2 literals: Let c = `1 ∨ `2 . Let u be a new variable. Consider
c0 = (`1 ∨ `2 ∨ u) ∧ (`1 ∨ `2 ∨ ¬u)
Again c is satisfiable iff c0 is satisfiable

21.1.2.4 SAT ≤P 3-SAT (contd)

Reduction Ideas: 2 and more literals

• Case clause with > 3 literals: Let c = `1 ∨ · · · ∨ `k . Let u1 , . . . uk−3 be new variables.
Consider
c0 = (`1 ∨ `2 ∨ u1 ) ∧ (`3 ∨ ¬u1 ∨ u2 ) ∧ (`4 ∨ ¬u2 ∨ u3 ) ∧
· · · ∧ (`k−2 ∨ ¬uk−4 ∨ uk−3 ) ∧ (`k−1 ∨ `k ∨ ¬uk−3 )
c is satisfiable iff c0 is satisfiable

Another way to see it — reduce size of clause by one:

c0 = (`1 ∨ `2 . . . ∨ `k−2 ∨ uk−3 ) ∧ (`k−1 ∨ `k ∨ ¬uk−3 )

21.1.2.5 An Example
Example 21.1.4 ϕ = (¬x1 ∨ ¬x4 ) ∧ (x1 ∨ ¬x2 ∨ ¬x3 ) ∧ (¬x2 ∨ ¬x3 ∨ x4 ∨ x1 ) ∧ (x1 ).
ψ = (¬x1 ∨ ¬x4 ∨ z) ∧ (¬x1 ∨ ¬x4 ∨ ¬z)
∧ (x1 ∨ ¬x2 ∨ ¬x3 )
∧ (¬x2 ∨ ¬x3 ∨ y1 ) ∧ (x4 ∨ x1 ∨ ¬y1 )
∧ (x1 ∨ u ∨ v) ∧ (x1 ∨ u ∨ ¬v) ∧ (x1 ∨ ¬u ∨ v) ∧ (x1 ∨ ¬u ∨ ¬v)

4
21.1.2.6 Overall Reduction Algorithm

Input: CNF formula ϕ

for each clause c of ϕ
if c does not have exactly 3 literals
construct c’ as before
else
c’ = c
ψ is conjunction of all c’ constructed in loop
is ψ satisfiable?

Correctness (informal)
ϕ is satisfiable iff ψ is satisfiable because for each clause c, the new 3CNF formula c0 is
logically equivalent to c.

21.1.2.7 What about 2-SAT?

2-SAT can be solved in polynomial time!
No known polynomial time reduction from SAT (or 3-SAT) to 2-SAT. If there was, then
SAT and 3-SAT would be solvable in polynomial time.

21.1.3 3-SAT and Independent Set

21.1.3.1 3-SAT ≤P Independent Set
Input Given a 3CNF formula ϕ

Goal Construct a graph Gϕ and number k such that Gϕ has an independent set of size k
iff ϕ is satisfiable. Gϕ should be constructible in time polynomial in size of ϕ

Importance of reduction: Although 3-SAT is much more expressive, it can be reduced to a

seemingly specialized Independent Set problem.

21.1.3.2 Interpreting 3-SAT

There are two ways to think about 3-SAT

• Find a way to assign 0/1 (false/true) to the variables such that the formula evaluates
to true, that is each clause evaluates to true

• Pick a literal from each clause and find a truth assignment to make all of them true.
You will fail if two of the literals you pick are in conflict, i.e., you pick xi and ¬xi

We will take the second view of 3-SAT to construct the reduction.

5
 ¬x1 ¬x2 ¬x1

x2 x3 x1 x3 x2 x4

Figure 21.1: Graph for ϕ = (¬x1 ∨ x2 ∨ x3 ) ∧ (x1 ∨ ¬x2 ∨ x3 ) ∧ (¬x1 ∨ x2 ∨ x4 )

21.1.3.3 The Reduction

• Gϕ will have one vertex for each literal in a clause
• Connect the 3 literals in a clause to form a triangle; the independent set will pick at
most one vertex from each clause, which will correspond to the literal to be set to true
• Connect 2 vertices if they label complementary literals; this ensures that the literals
corresponding to the independent set do not have a conflict
• Take k to be the number of clauses

21.1.3.4 Correctness
Proposition 21.1.5 ϕ is satisfiable iff Gϕ has an independent set of size k (= number of
clauses in ϕ).

Proof :
⇒ Let a be the truth assignment satisfying ϕ
– Pick one of the vertices, corresponding to true literals under a, from each triangle.
This is an independent set of the appropriate size

21.1.3.5 Correctness (contd)

Proposition 21.1.6 ϕ is satisfiable iff Gϕ has an independent set of size k (= number of
clauses in ϕ).

Proof :
⇐ Let S be an independent set of size k
– S must contain exactly one vertex from each clause
– S cannot contain vertices labeled by conflicting clauses
– Thus, it is possible to obtain a truth assignment that makes in the literals in S
true; such an assignment satisfies one literal in every clause

6
21.1.3.6 Transitivity of Reductions
X ≤P Y and Y ≤P Z implies that X ≤P Z.
Note: X ≤P Y does not imply that Y ≤P X and hence it is very important to know the
FROM and TO in a reduction.

To prove X ≤P Y you need to show a reduction FROM X TO Y

In other words show that an algorithm for Y implies an algorithm for X.

21.2 Definition of NP
21.2.0.7 Recap . . .

Problems
• Independent Set
• Vertex Cover
• Set Cover
• SAT
• 3-SAT

Relationship
≤P
3-SAT ≤P Independent Set ≥P Vertex Cover ≤P Set Cover
3-SAT ≤P SAT ≤P 3-SAT

21.3 Preliminaries
21.3.1 Problems and Algorithms
21.3.1.1 Problems and Algorithms: Formal Approach

Decision Problems
• Problem Instance: Binary string s, with size |s|
• Problem: A set X of strings on which the answer should be “yes”; we call these YES
instances of X. Strings not in X are NO instances of X.

Definition 21.3.1 • A is an algorithm for problem X if A(s) = ”yes” iff s ∈ X

• A is said to have a polynomial running time if there is a polynomial p(·) such that for
every string s, A(s) terminates in at most O(p(|s|)) steps

7
21.3.1.2 Polynomial Time
Definition 21.3.2 Polynomial time (denoted P ) is the class of all (decision) problems that
have an algorithm that solves it in polynomial time

Example 21.3.3 ¡2-¿ Problems in P include

• Is there a shortest path from s to t of length ≤ k in G?

• Is there a flow of value ≥ k in network G?

• Is there an assignment to variables to satisfy given linear constraints?

21.3.1.3 Efficiency Hypothesis

A problem X has an efficient algorithm iff X ∈ P , that is X has a polynomial time algorithm.
Justifications:

• robustness of definition to variations in machines

• a sound theoretical definition

• most known polynomial time algorithms for “natural” problems have small polynomial
running times

21.3.1.4 Problems with no known polynomial time algorithms

Problems

• Independent Set

• Vertex Cover

• Set Cover

• SAT

• 3-SAT

There are of course undecidable problems (no algorithm at all!) but many problems that
we want to solve are like above.
Question: What is common to above problems?

8
21.3.1.5 Efficient Checkability

Above problems share the following feature:

For any YES instance IX of X there is a proof/certificate/solution that is of length

poly(|IX |) such that given a proof one can efficiently check that IX is indeed a YES instance
Examples:

• SAT formula ϕ: proof is a satisfying assignment

• Independent Set in graph G and k: a subset S of vertices

21.3.2 Certifiers/Verifiers
21.3.2.1 Certifiers

Definition 21.3.4 An algorithm C(·, ·) is a certifier for problem X if for every s ∈ X there
is some string t such that C(s, t) =”yes”, and conversely, if for some s and t, C(s, t) =”yes”
then s ∈ X.
The string t is called a certificate or proof for s

Efficient Certifier
C is an efficient certifier for problem X if there is a polynomial p(·) such that for every string
s, s ∈ X iff there is a string t with |t| ≤ p(|s|), C(s, t) = ”yes” and C runs in polynomial
time

21.3.2.2 Example: Independent Set

• Problem: Does G = (V, E) have an independent set of size ≥ k?

– Certificate: Set S ⊆ V
– Certifier: Check |S| ≥ k and no pair of vertices in S is connected by an edge

21.3.3 Examples
21.3.3.1 Example: Vertex Cover

• Problem: Does G have a vertex cover of size ≤ k?

– Certificate: S ⊆ V
– Certifier: Check |S| ≤ k and that for every edge at least one endpoint is in S

9
21.3.3.2 Example: SAT
• Problem: Does formula ϕ have a satisfying truth assignment?

– Certificate: Assignment a of 0/1 values to each variable

– Certifier: Check each clause under a and say “yes” if all clauses are true

21.3.3.3 Example:Composites
• Problem: Is number s a composite?

– Certificate: A factor t ≤ s such that t 6= 1 and t 6= s

– Certifier: Check that t divides s (Euclid’s algorithm)

21.4 NP
21.4.1 Definition
21.4.1.1 Nondeterministic Polynomial Time
Definition 21.4.1 Nondeterministic Polynomial Time (denoted by N P ) is the class of all
problems that have efficient certifiers

Example 21.4.2 ¡2-¿ Independent Set, Vertex Cover, Set Cover, SAT, 3-SAT, Composites
are all examples of problems in N P

21.4.1.2 Asymmetry in Definition of NP

Note that only YES instances have a short proof/certificate. NO instances need not have a
short certificate.

Example: SAT formula ϕ. No easy way to prove that ϕ is NOT satisfiable!

More on this and co-NP later on.

21.4.2 Intractibility
21.4.2.1 P versus N P
Proposition 21.4.3 P ⊆ N P
For a problem in P no need for a certificate!

Proof : Consider problem X ∈ P with algorithm A. Need to demonstrate that X has an

efficient certifier

10
 • Certifier C on input s, t, runs A(s) and returns the answer

• C runs in polynomial time

• If s ∈ X then for every t, C(s, t) = ”yes”

• If s 6∈ X then for every t, C(s, t) = ”no”

21.4.2.2 Exponential Time

Definition 21.4.4 Exponential Time (denoted EXP ) is the collection of all problems that
have an algorithm which on input s runs in exponential time, i.e., O(2poly(|s|) )
3
Example: O(2n ), O(2n log n ), O(2n ), ...

21.4.2.3 N P versus EXP

Proposition 21.4.5 N P ⊆ EXP

Proof : Let X ∈ N P with certifier C. Need to design an exponential time algorithm for X

• For every t, with |t| ≤ p(|s|) run C(s, t); answer “yes” if any one of these calls returns
“yes”

• The above algorithm correctly solves X (exercise)

• Algorithm runs in O(q(|s| + |p(s)|)2p(|s|) ), where q is the running time of C

21.4.2.4 Examples
• SAT: try all possible truth assignment to variables

• Independent set: try all possible subsets of vertices

• Vertex cover: try all possible subsets of vertices

21.4.2.5 Is N P efficiently solvable?

We know P ⊆ N P ⊆ EXP
Big Question
Is there are problem in N P that does not belong to P ? Is P = N P ?

11