Complexity classes

1
 Optimization & Decision Problems
• Decision problems
– Given an input and a question regarding a problem,
determine if the answer is yes or no
• Optimization problems
– Find a solution with the “best” value
• Optimization problems can be cast as decision
problems that are easier to study
– E.g.: Shortest path: G = unweighted directed graph
• Find a path between u and v that uses the fewest
edges
• Does a path exist from u to v consisting of at most k
edges? 2
Algorithmic vs Problem Complexity
• The algorithmic complexity of a computation is
some measure of how difficult is to perform the
computation (i.e., specific to an algorithm)
• The complexity of a computational problem
or task is the complexity of the algorithm with the
lowest order of growth of complexity for solving
that problem or performing that task.
– e.g. the problem of searching an ordered list has at
most lgn time complexity.
• Computational Complexity: deals with
classifying problems by how hard they are.
3
 Class of “P” Problems
• Class P consists of (decision) problems that are
solvable in polynomial time
• Polynomial-time algorithms
– Worst-case running time is O(nk), for some constant k
• Examples of polynomial time:
– O(n2), O(n3), O(1), O(n lg n)
• Examples of non-polynomial time:
– O(2n), O(nn), O(n!)

4
Polynomial and Exponential
Polynomial Time Exponential Time
Linear Search 0/1 Knapsack
Binary Search Travelling Salesman
Problem
Matrix Multiplication Hamiltonian Cycle
Sorting Graph Colouring
Fractional Knapsack Sum of Subset

5
 Tractable/Intractable Problems
• Problems in P are also called tractable
• Problems not in P are intractable or unsolvable
– Can be solved in reasonable time only for small inputs
– Or, can not be solved at all

• Are non-polynomial algorithms always worst than

polynomial algorithms?
- n1,000,000 is technically tractable, but really impossible -
nlog log log n is technically intractable, but easy

6
 Example of Unsolvable Problem
• Turing discovered in the 1930’s that there are
problems unsolvable by any algorithm.
• The most famous of them is the halting
problem
– Given an arbitrary algorithm and its input, will that
algorithm eventually halt, or will it continue forever in
an “infinite loop?”

7
Examples of Intractable Problems

8
 Intractable Problems
• Can be classified in various categories based on
their degree of difficulty, e.g.,
– NP
– NP-complete
– NP-hard
• Let’s define NP algorithms and NP problems …

9
 Nondeterministic and NP Algorithms

Nondeterministic algorithm = two stage procedure:

1) Nondeterministic (“guessing”) stage:
generate randomly an arbitrary string that can be
thought of as a candidate solution (“certificate”)

2) Deterministic (“verification”) stage:

take the certificate and the instance to the problem and
returns YES if the certificate represents a solution

NP algorithms (Nondeterministic polynomial)

verification stage is polynomial
10
 Class of “NP” Problems
• Class NP consists of problems that could be
solved by NP algorithms
– i.e., verifiable in polynomial time

• If we were given a “certificate” of a solution, we

could verify that the certificate is correct in time
polynomial to the size of the input
• Warning: NP does not mean “non-polynomial”

11
 E.g.: Hamiltonian Cycle
• Given: a directed graph G = (V, E), determine a
simple cycle that contains each vertex in V
– Each vertex can only be visited once
• Certificate:
– Sequence: v1, v2, v3, …, v|V| hamiltonian

not
hamiltonian

12
 Is P = NP?
• Any problem in P is also in NP: P

P  NP NP

• The big (and open question) is whether NP  P

or P = NP
– i.e., if it is always easy to check a solution, should it
also be easy to find a solution?

• Most computer scientists believe that this is

false but we do not have a proof …
13
 NP-Completeness (informally)

• NP-complete problems are P

NP-complete

defined as the hardest NP

problems in NP
• Most practical problems turn out to be either P or
NP-complete.
• Study NP-complete problems …

14
 Reductions
• Reduction is a way of saying that one problem is
“easier” than another.
• We say that problem A is easier than problem B,
(i.e., we write “A  B”)
if we can solve A using the algorithm that solves B.
• Idea: transform the inputs of A to inputs of B

yes
  yes
f Problem B no
no
Problem A
15
 Polynomial Reductions

• Given two problems A, B, we say that A is

polynomially reducible to B (A p B) if:

1. There exists a function f that converts the input of A

to inputs of B in polynomial time

2. A(i) = YES  B(f(i)) = YES

16
 NP-Completeness (formally)
• A problem B is NP-complete if:
P NP-complete
(1) B  NP
NP
(2) A p B for all A  NP

• If B satisfies only property (2) we say that B is NP-hard

• No polynomial time algorithm has been discovered for an

NP-Complete problem
• No one has ever proven that no polynomial time
algorithm can exist for any NP-Complete problem
17
 Implications of Reduction

yes
  yes
f Problem B no
no
Problem A

- If A p B and B  P, then A  P

- if A p B and A  P, then B  P

18
 Proving Polynomial Time

yes
  Polynomial time
yes
f algorithm to decide B no
no
Polynomial time algorithm to decide A

1. Use a polynomial time reduction algorithm to

transform A into B
2. Run a known polynomial time algorithm for B
3. Use the answer for B as the answer for A

19
 Proving NP-Completeness In Practice

• Prove that the problem B is in NP

– A randomly generated string can be checked in
polynomial time to determine if it represents a solution

• Show that one known NP-Complete problem

can be transformed to B in polynomial time
– No need to check that all NP-Complete problems are
reducible to B

21
 Revisit “Is P = NP?”

P NP-complete

NP

Theorem: If any NP-Complete problem can be

solved in polynomial time  then P = NP.

22
 P & NP-Complete Problems

• Shortest simple path

– Given a graph G = (V, E) find a shortest path from a
source to all other vertices
– Polynomial solution: O(VE)

• Longest simple path

– Given a graph G = (V, E) find a longest path from a
source to all other vertices
– NP-complete
23
 P & NP-Complete Problems
• Euler tour
– G = (V, E) a connected, directed graph find a cycle
that traverses each edge of G exactly once (may visit
a vertex multiple times)
– Polynomial solution O(E)

• Hamiltonian cycle
– G = (V, E) a connected, directed graph find a cycle
that visits each vertex of G exactly once
– NP-complete

24
 Satisfiability Problem (SAT)
• Satisfiability problem: given a logical
expression , find an assignment of values
(F, T) to variables xi that causes  to evaluate
to T
 =x1   x2  x3   x4

• SAT was the first problem shown to be NP-

complete!

25
 CFN Satisfiability
• CFN is a special case of SAT
  is in “Conjuctive Normal Form” (CNF)
– “AND” of expressions (i.e., clauses)
– Each clause contains only “OR”s of the variables and
their complements

E.g.:  = (x1  x2)  (x1   x2)  ( x1   x2)

clauses
26
 3-CNF Satisfiability
A subcase of CNF problem:
– Contains three clauses
• E.g.:
 = (x1  x1  x2)  (x3  x2  x4) 
(x1  x3   x4)

• 3-CNF is NP-Complete

• Interestingly enough, 2-CNF is in P!

27
 Clique
Clique Problem:
– Undirected graph G = (V, E)
– Clique: a subset of vertices in V all connected to each
other by edges in E (i.e., forming a complete graph)
– Size of a clique: number of vertices it contains
Clique(G, 2) = YES
Optimization problem: Clique(G, 3) = NO
– Find a clique of maximum size
Decision problem:
Clique(G, 3) = YES
– Does G have a clique of size k? Clique(G, 4) = NO

28
 Clique Verifier
• Given: an undirected graph G = (V, E)
• Problem: Does G have a clique of size k?
• Certificate:
– A set of k nodes

• Verifier:
– Verify that for all pairs of vertices in this set there
exists an edge in E

29
 3-CNF p Clique

• Idea:

– Construct a graph G such that  is satisfiable only if

G has a clique of size k

30
 NP-naming convention
• NP-complete - means problems that are
'complete' in NP, i.e. the most difficult to solve in
NP
• NP-hard - stands for 'at least' as hard as NP
(but not necessarily in NP);
• NP-easy - stands for 'at most' as hard as NP
(but not necessarily in NP);
• NP-equivalent - means equally difficult as NP,
(but not necessarily in NP);

31
Examples NP-complete and
NP-hard problems

NP-complete

NP-hard

32
 Difference

NP-hard NP-Complete
NP-Hard problems(say X) can NP-Complete problems can be
be solved if and only if there is a solved by a non-deterministic
NP-Complete problem(say Y) Algorithm/Turing Machine in
that can be reducible into X in polynomial time.
polynomial time
To solve this problem, it do not To solve this problem, it must be
have to be in NP . both NP and NP-hard problems.
Not all NP-hard problems are All NP-complete problems are
NP-complete. NP-hard
Example: Halting problem, Example: Determine whether a
Vertex cover problem, etc. graph has a Hamiltonian cycle,
Determine whether a Boolean
formula is satisfiable or not,
Circuit-satisfiability problem, etc.

33
Examples NP-complete

https://www.wikiwand.com/en/List_of_NP-
complete_problems

34