CSE 317: Design and Analysis of Algorithms

Shahid Hussain
Weeks 14 and 16: November 18 - November 27, 2024: Fall 2024

1
Computational Complexity
Decision Problems vs Optimization Problems

Decision Problem
A computational problem is called decision problem if the answer is either yes
or no. For example:
• Is there a path of length k in a graph G?
• Is a given array A sorted in non-decreasing order?

Optimization Problem
A computational problem is called optimization problem if the answer is a value
that needs to be maximized or minimized. For example:
• Find the shortest path between two nodes in a graph G.
• Find the minimum spanning tree of a graph G.

2
Computational Complexity

• Computational complexity is a field that deals with measuring the

resources required to solve a computational problem and classifying problems
based on their resource requirements
• The most common resource is time required to solve a problem
• Often we are only interested in the decision problems and we are interested in
the time required to solve the problem

3
Model of Computation

• We need a model of computation to measure the time required to solve a

problem
• The most common model of computation is the Turing machine
• A Turing machine is an abstract machine that can simulate any algorithm
• The time complexity of an algorithm is measured in terms of the number of
steps required to solve a problem

4
Languages and Decision Problems

Language
A language is a set of strings over a finite alphabet (Σ) For example, the set of all
binary strings that represent prime numbers is a language

• We can now redefine the decision problem as follows:

• Decision Problem: Given a string x ∈ Σ∗ , is x in the language L?
• That is., a decision problem is a kind of membership query for a string in a
language

5
Algorithms and Machines

• Let Σ be a finite alphabet and A be an algorithm (machine) that implements

a mapping
φ : Σ∗ → {0, 1}

• We say A is a polynomial time algorithm if there exists a polynomial p(n)

such that for all x ∈ Σ∗ , the algorithm A halts in at most p(|x|) steps
•

6
The Class P

The class P
The class P contains all languages (decision problems) which can be decided by
some polynomial time algorithm i.e., the membership queries can be computed in
polynomial time by some algorithm

7
The Class NP

• Let Σ be a finite alphabet

• We say Q : Σ∗ → {0, 1} is a polynomial time predicate if there exists a
polynomial time algorithm that for given words α, β ∈ Σ∗ , compute Q(α, β)
The class NP
We say that a language L ⊆ Σ∗ is in the class NP if and only if there exists a
polynomial q and a polynomial time predicate Q such that for any word α ∈ Σ∗ ,
we have α ∈ L if and only if there exists a word β ∈ Σ∗ such that Q(α, β) = 1 and
|β| ≤ q(|α|) in other words:

L ⊆ NP ⇐⇒ ∃q ∃Q ∀α ∈ Σ∗ ∃β ∈ Σ∗ Q(α, β) = 1, s.t. |β| ≤ q(|α|)

The word β is called the certificate and the algorithm which computes the
predicate Q is called the verifier.
8
The Classes P and NP

Theorem
P ⊆ NP.

Proof.
• Let L ∈ P, L ⊆ Σ∗ , and q(n) = 1
• Let us consider a predicate Q(α, β) such that Q(α, β) = 1 if and only if α ∈ L
• Since L ∈ P, the predicate Q is a polynomial time predicate
• It is clear that α ∈ L if and only if there exists β ∈ Σ∗ such that |β| ≤ 1 and
Q(α, β) = 1 (we can take β = ϵ)

9
Reduction

Polynomial time reduction

We say that a language L1 ⊆ Σ∗ is a polynomial-time reducible to a language
L2 ∈ Σ∗ (written as L1 ≤P L2 ) if there exists a polynomial time computable
function φ : Σ∗ → Σ∗ such that, for all α we have α ∈ L1 if and only if φ(α) ∈ L2 .

Proposition
1. If L1 ≤P L2 and L2 ∈ P then L1 ∈ P.
2. If L1 ≤P L2 and L1 ̸∈ P then L2 ̸∈ P.
3. If L1 ≤P L2 and L2 ≤P L3 then L1 ≤P L3 . (Transitivity property.)

10
NP-hard and NP-complete Problems

NP-hard problems
A language L is NP-hard if for all L′ ∈ NP, we have L′ ≤P L.

NP-complete problems
A language L is NP-complete if L ∈ NP and L is NP-hard.

• If L is NP-hard and L ∈ P, then P = NP.

• If L is NP-hard and P ̸= NP, then L ̸∈ P.

11
Boolean Functions and CNF

Boolean Function/Formula
A function f : {0, 1}n → {0, 1} is called a boolean function of n variables.

• A boolean formula of the following kind is called a conjunctive normal form

(CNF):
f (x1 , x2 , . . . , xn ) = C1 ∧ C2 ∧ . . . ∧ Cm

where each Cj is a clause of the form lj1 ∨ lj2 ∨ . . . ∨ ljk and each lji is a literal
which is either a variable xi or its negation ¬xi
• We assume that each variable occurs in each clause Cj at most once

12
Satisfiability Problem

Satisfiability Problem
Decision Problem: sat
Setup: A boolean formula f of n variables in CNF
Question: Is there an assignment of values to the variables such that f evaluates
to true?

• We show that sat is NP-complete

• We need to show that sat is in NP and it is NP-hard

13
Satisfiability Problem is in NP

• Given a boolean formula f in CNF, we can verify in polynomial time whether

a given assignment of values to the variables makes f evaluate to true
• We can verify this by checking each clause Cj and checking if at least one
literal in the clause evaluates to true
• Thus, sat is in NP

14
Satisfiability Problem is NP-hard

Theorem (Cook-Levin 1971)

sat is NP-complete.

• This proof was independently discovered by Stephen Cook (in United States)
and Leonid Levin (in, then, Soviet Union) in 1971
• This proof laid down the theoretical/mathematical foundation of the theory of
NP-complete problems
• In the subsequent we present some NP-complete problems all of which can
proved to be NP-hard using the polynomial time reduction (due to transitivity
property of reduction)

15
The 3 SAT problem

The 3 SAT problem

Decision Problem: 3sat
Setup: A boolean formula f of n variables in CNF where each clause has at most
3 literals
Question: Is there an assignment of values to the variables such that f evaluates
to true?

• 3sat ∈ NP (trivial)
• sat ≤P 3sat (left as an exercise)

16
Independent Set Problem

Independent Set Problem

Let G = (V, E) be an undirected graph. We say a subset of vertices S ⊆ V is
independent set if no two vertices in S are adjacent. There is no edge that joins
any two vertices in S.
The optimization version of the independent set problem is to find an independent
set of maximum cardinality in G.
Decision Problem: is
Setup: An undirected graph G = (V, E) and an integer k
Question: Is there an independent set of size at least k in G?

17
Independent Set Problem

18
Proving Independent Set to NP-complete

1. We need to show that is ∈ NP

2. We show that for some NP-complete problem Π, Π ≤P is

19
Showing Independent Set is in NP

• Given a graph G = (V, E) and an integer k, we can verify in polynomial time

whether a given subset of vertices S ⊆ V is an independent set of size at
least k
• We can verify this by checking if no two vertices in S are adjacent and if
|S| ≥ k
• Thus, is is in NP

20
Showing Independent Set is NP-hard

• We show that is is NP-hard by showing that 3sat ≤P is

• Given a boolean formula f = C1 ∧ C2 ∧ . . . ∧ Cm in 3CNF, where each
Cj = lj1 ∨ lj2 ∨ lj3 , we construct a graph G = (V, E) such that f is satisfiable
if and only if G has an independent set of size at least k
• We construct the graph G as follows:
• Each literal in each clause has a corresponding vertex in G
• Each clause Cj has a corresponding triangle in G
• Each vertex v from each clause is connected to ¬v from other clauses and vice
versa
• We set k = m
• We show that f is satisfiable if and only if G has an independent set of size at
least k

21
Example Reduction

• Polynomial-time reduction of 3sat to is

f = (x1 ∨ ¬x2 ∨ x3 ) ∧ (¬x1 ∨ ¬x3 ) ∧ (x1 ∨ x3 )

¬x2 ¬x1 x1

x1

x3 ¬x3 x3

• Above formula is satisfied: x1 = 1, x2 = 0, x3 = 0

22
Vertex Cover Problem

Vertex Cover Problem

Let G = (V, E) be an undirected graph. We say a subset of vertices S ⊆ V is a
vertex cover if every edge in E has one endpoint in S.
The optimization version of the vertex cover problem is to find a vertex cover of
minimum cardinality in G.
Decision Problem: vc
Setup: An undirected graph G = (V, E) and an integer k
Question: Is there a vertex cover of size at most k in G?

23
Vertex Cover Problem

24
Proving Vertex Cover is NP-complete

• We show that vc is NP-hard by showing that is ≤P vc

• Clearly, if S is an independent set in G, then V \ S is a vertex cover in G

– Vertex cover

– Independent set

• If S is an independent set, each e ∈ E has at least one endpoint is in V \ S

• Therefore, V \ S is a vertex cover in G
• It implies that G has an independent set with at least k vertices if and only if
G has a vertex cover with at most n − k vertices
• Therefore, vc is NP-complete
25
Set Cover Problem

Set Cover Problem

S
Let U be a set and F a family of subsets of U s.t., U = S∈F S.
The optimization version of the set cover problem is to find a set cover of minimum
cardinality in F.
Decision Problem: sc
Setup: A finite set U, a collection of subsets F of U, and an integer k
Question: Is there a set cover of size at most k in F?

26
Set Cover Problem

• Let U = {a1 , a2 , a3 , a4 , a5 , a6 } and F = {S1 , S2 , S3 , S4 }

• S1 = {a1 , a2 , a3 }, S2 = {a1 , a4 }, S3 = {a2 , a5 }, S4 = {a3 , a6 }

a1 a2 a3 S1

a4 a5 a6

S2 S3 S4

27
Set Cover Problem

• It is easy to show that sc is in NP

• We show that vc ≤P sc
• Let G = (V, E) be an undirected graph and k be an integer
• We construct a set cover problem (UG , FG ) as follows:
• Let UG = E and FG = {Sv | v ∈ V }
• Where Sv is the set of edges from E that have v as an endpoint
• We set k = |V |
• We show that {vi1 , . . . , vik } is a vertex cover for G if and only if:
[
Sv = UG
Sv ∈{Svi ,...,Svi }
1 k

• Thus, sc is NP-complete

28
Set Cover Problem

d
2 4
a h
e g
1 c 6

b 3 5
k
f

• The set cover problem UG , FG for the above graph G = (V, E) is:
• UG = {a, b, c, d, e, f, g, h, k}
• FG = {S1 , S2 , S3 , S4 , S5 , S6 }
• S1 = {a, d}, S2 = {a, c}, S3 = {d, e}, S4 = {d, h}, S5 = {e, g}, S6 = {h, k}

29